Method for incorporating facial recognition technology in a multimedia surveillance system

• US 7,634,662 B2
• Filed: 11/21/2003
• Issued: 12/15/2009
• Est. Priority Date: 11/21/2002
• Status: Expired due to Fees

• Current Assignee: e-Watch, Inc. (Cision Ltd.)
• Original Assignee: e-Watch, Inc. (Cision Ltd.)
• Inventors: Monroe, David A.
• Primary Examiner(s): Cervetti; David García
• Assistant Examiner(s): Traore; Fatoumata
• Application Number:
  US10/719,792
• Publication Number:
  US 20040117638A1
• Time in Patent Office: 2,216 Days
• Field of Search: 713/186
• US Class Current: 713/186
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 2201/05 Recognition of patterns representing particular kinds of hidden objects, e.g. weapons, explosives, drugs
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G07C 1/20 Checking timed patrols, e.g. of watchman
• CPC Class Codes: G07C 2011/02 related to amusement parks
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition
• CPC Class Codes: G07C 9/38 with central registration
• CPC Class Codes: G08B 13/1965 the vehicle being an aircraft
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound
• CPC Class Codes: G08B 25/016 Personal emergency signalling and security systems emergency non-personal manually actuated alarm activators G08B25/12
• CPC Class Codes: G08B 25/14 Central alarm receiver or annunciator arrangements

Abstract

Embodiment provide a surveillance camera adapted to be connected to an internet protocol network, the surveillance camera including at least one facial processor, at least one facial recognition algorithm embodied in suitable media, at least one facial recognition algorithm executable with digital format image data by at least one facial processor detecting faces, execution of at least one facial recognition algorithm producing unique facial image data, execution of at least one facial separation algorithm producing facial separation data, at least one facial processor in communication with at least one facial signature database to obtain reference data, execution of at least one facial signature algorithm comparing facial separation data and reference data to identify correlations, at least one compression algorithm producing compressed image data, and a network stack configured to transmit to the network unique facial image data for each detected face and compressed image data.

Claims

1. A surveillance system having at least one camera adapted to produce an IP signal, the IP signal being transmitted over an IP network, the system comprising:

the at least one camera having an image collection device configured for collecting image data;

the at least one camera having an analog to digital converter in communication with the image collection device for convening collected image data from analog format to digital format image data;

the at least one camera having at least one facial processor in communication with the analog to digital converter for receiving the digital format image data, the at least one facial processor being configured to execute with the digital format image data at least one facial recognition algorithm, execution of the at least one facial recognition algorithm with the digital format image data detecting faces when present in the digital format image data, execution of the at least one facial recognition algorithm providing for each detected face at least one set of unique facial image data;

the at least one camera having at least one compressor in communication with the analog to digital converter for receiving the digital format image data, the at least one compressor being configured to execute with the digital format image data at least one compression algorithm, execution of the at least one compression algorithm with the digital format image data providing at least one set of compressed image data;

the at least one camera having a network protocol stack, the network protocol stack being configured for transmitting to the IP network for each detected face the at least one set of unique facial image data, the network protocol stack being configured for transmitting to the IP network the at least one set of compressed image data; and

at least one facial signature processor in communication with the IP network, the at least one facial signature processor being remote from the at least one camera, the at least one facial signature processor being configured to receive from the IP network for each detected face the at least one set of unique facial image data, the at least one facial signature processor being in communication with at least one facial recognition database, the at least one facial signature processor being configured to compare with the at least one facial recognition database the at least one set of unique facial image data of each detected face;

wherein the at least one set of unique facial image data for each detected face is provided from the at least one camera to the IP network, such that the at least one set of unique facial image data of each detected face is received by the at least one facial signature processor to be compared with the at least one facial recognition database without preceding transmission of the digital format image data from the at least one camera over the IP network.

2. The system of claim 1, further comprising:

a server in communication with the IP network, the server being remote from the camera, the server receiving the at least one set of unique facial image data for each detected face.

3. The system of claim 2 and further comprising:

the server being in communication with the at least one facial image database, the server being in communication with the at least one facial signature processor, the at least one facial signature processor being configured to compare the at least one set of facial image data with the at least one facial image database.

4. The system of claim 2 and further comprising:

the server being in communication with a third party database for at least one of;

sending, receiving, and both sending and receiving facial image data to a third party.

5. The system of claim 2 and further comprising:

a storage device in communication with the IP network for archiving archival data, the archival data including least one of;

the at least one set of facial image data and the at least one set of compressed image data.

6. The system of claim 5 and further comprising:

a data mining system in communication with the IP network for mining the archival data.

7. The system of claim 1 and further comprising:

the at least one facial recognition algorithm including at least one facial separation algorithm, the at least one facial separation algorithm when executed producing at least one set of facial separation data, the at least one set of facial image data including the at least one set of facial separation data.

8. The system of claim 1 and further comprising:

a plurality of cameras in communication with the IP network for collecting image data at distributed locations.

9. The system of claim 8 and further comprising:

a server in communication with the IP network, the server being remote from the plurality of cameras, the server receiving from each camera respective of the at least one set of unique facial image data of each detected face.

10. The system of claim 1 and further comprising:

a remote station in communication with the IP network, the remote station receiving the at the at least one set of compressed image data, the remote station being configured to display the at least one set of compressed image data.

11. The system of claim 10 and further comprising:

the remote station receiving the at least one set of unique facial image data of each detected face, the remote station being in communication with at least one facial image database, the remote station being in communication with at least one facial signature processor, the at least one facial signature processor being configured to compare the at least one set of unique facial image data of each detected face with the at least one facial image database, the remote station being configured to display a result provided by the at least one facial image processor.

12. The system of claim 10 and further comprising:

wherein the remote station includes at least one of;

a desktop computer, a portable computer, a PDA, and a wireless device.

13. The system of claim 10 and further comprising:

the remote station further including a system map database and a display monitor for displaying the system map, the system map including an icon for identifying the location of the camera.

14. The system of claim 13 and further comprising:

a plurality of cameras in communication with the IP network, the remote station displaying an identifying icon for each of said cameras.

15. The system of claim 14 and further comprising:

a tracking system in communication with the remote station for tracking the progress of an individual as he moves from a field of view of a camera to a field of view of a subsequent camera.

16. The system of claim 1 and further comprising:

an access control device in communication with the IP network, the access control device being responsive to an activation signal, the control device being activated upon confirmation of identity between the at least one set of facial image data and data in a facial image database.

17. A surveillance camera adapted to be connected to an internet protocol network, the surveillance camera comprising:

an image collection device, the image collection device being configured to collect image data;

at least one analog to digital converter in communication with the image collection device for convening collected image data from analog format to digital format image data;

at least one facial processor in communication with the analog to digital converter to receive the digital format image data;

at least one facial recognition algorithm embodied in suitable media, the at least one facial recognition algorithm being executable with the digital format image data by the at least one facial processor, execution of the at least one facial recognition algorithm with the digital format image data detecting faces when present in the digital format image data, execution of the at least one facial recognition algorithm producing for each detected face at least one set of unique facial image data, the at least one facial recognition algorithm including at least one facial separation algorithm, execution of the at least one facial separation algorithm with the digital format data producing at least one set of facial separation data corresponding to the digital format image data, the at least one facial recognition algorithm including at least one facial signature algorithm;

the at least one facial processor being in communication with at least one facial signature database to obtain from the at least one facial signature database a plurality of sets of reference facial separation data, execution of the at least one facial signature algorithm comparing the at least one set of facial separation data and the plurality of sets of reference facial separation data to identify correlations between the at least one set of facial separation data and the plurality of sets of reference facial separation data;

at least one compression algorithm embodied in suitable media, the at least one compression algorithm being executable with the digital format image data by at least one compressor, execution of the at least one compression algorithm producing at least one set of compressed image data; and

a network stack in communication with the at least one compressor, the network stack being configured to transmit to the internet protocol network the at least one set of unique facial image data for each detected face the network stack being configured to transmit to the internet protocol network the at least one set of compressed image data;

wherein the at least one set of unique facial image data for each detected face can be provided from the at least one camera to the IP network.

18. A camera according to claim 17 and further comprising:

a housing, the housing commonly supporting the image collection device, the at least one analog to digital converter, the at least one facial recognition algorithm, the at least one facial processor, the at least one compression algorithm, the at least one compressor, and the internet protocol network stack.

19. A camera according to claim 17 and further comprising:

wherein the at least one facial signature database is stored in local media, the local media being located in the camera.

20. A camera according to claim 17 and further comprising:

wherein the at least one facial signature database is stored in remote media at a location remote from the camera, the remote media being in communication with the internet protocol network, the plurality of sets of reference facial separation data being provided from the remote media to the camera over the internet protocol network.

21. A camera according to claim 17 and further comprising:

the at least one set of compressed image data including at least one set of low resolution compressed image data having a respective low resolution and at least one set of high resolution compressed image data having a respective high resolution, the low resolution being less than the high resolution.

22. A camera according to claim 21 and further comprising:

the at least one set of low resolution compressed image data including MPEG data, the at least one set of high resolution compressed image data including JPEG data.

23. A camera according to claim 22 and further comprising:

the network stack transmitting the at least one set of low resolution compressed image data including MPEG data according to multicast protocol, the network stack transmitting the at least one set of high resolution compressed image data including JPEG data according to one of;

multicast protocol and unicast protocol.

24. A camera according to claim 21 and further comprising:

the network stack transmitting the at least one set of low resolution compressed image data according to multicast protocol.

25. A camera according to claim 24 and further comprising:

the network stack transmitting the at least one set of high resolution compressed image data according to one of;

multicast protocol and unicast protocol.

26. A camera according to claim 17 and further comprising:

the network stack transmitting a portion of the at least one set of compressed image data according to multicast protocol.

27. A surveillance camera adapted to be connected to an internet protocol network, the surveillance camera comprising:

an image collection device, the image collection device being configured to collect image data;

at least one analog to digital converter in communication with the image collection device for convening collected image data from analog format to digital format image data;

at least one facial processor in communication with the analog to digital converter to receive the digital format image data;

at least one facial recognition algorithm embodied in suitable media, the at least one facial recognition algorithm being executable with the digital format image data by the at least one facial processor, execution of the at least one facial recognition algorithm with the digital format image data detecting faces when present in the digital format image data, execution of the at least one facial recognition algorithm producing for each detected face at least one set of unique facial image data, the at least one facial recognition algorithm including at least one facial separation algorithm, execution of the at least one facial separation algorithm with the digital format data producing at least one set of facial separation data corresponding to the digital format image data;

at least one compression algorithm embodied in suitable media, the at least one compression algorithm being executable with the digital format image data by at least one compressor, execution of the at least one compression algorithm producing at least one set of compressed image data; and

a network stack in communication with the at least one compressor, the network stack being configured to transmit to the internet protocol network the at least one set of unique facial image data for each detected face the network stack being configured to transmit to the internet protocol network the at least one set of compressed image data;

the network stack transmitting the at least one set of facial separation data according to multicast protocol to a remote recipient, the remote recipient being in communication with the internet protocol network, the remote recipient being in communication with at least one remote facial signature processor, the at least one remote facial signature processor being associated with suitable media embodying at least one facial signature algorithm, the at least one remote facial signature processor being configured to execute the at least one facial signature algorithm, the at least one remote facial signature processor being in communication with at least one facial signature database to obtain from the at least one facial signature database a plurality of sets of reference facial separation data, the at least one facial signature algorithm when executed comparing the at least one set of facial separation data and the plurality of sets of reference facial separation data to identify correlations between the at least one set of facial separation data and the plurality of sets of reference facial separation data.

28. A surveillance camera adapted to be connected to an internet protocol network, the surveillance camera comprising:

an image collection device, the image collection device being configured to collect image data of a field of view, the image collection device providing digital format image data;

at least one processor in communication with an image collection device to receive the digital format image data;

at least one facial recognition algorithm embodied in suitable media, the at least one facial recognition algorithm being executable with the digital format image data by the at least one facial processor, execution of the at least one facial recognition algorithm detecting faces when present in the digital image format data, execution of the at least one facial recognition algorithm producing at least one set of unique facial image data for each detected face, the at least one facial recognition algorithm including at least one facial separation algorithm, execution of the at least one facial separation algorithm with the digital format data producing at least one set of facial separation data corresponding to the digital format image data, the at least one facial recognition algorithm including at least one facial signature algorithm;

the at least one facial processor being in communication with at least one facial signature database to obtain from the at least one facial signature database a plurality of sets of reference facial separation data, execution of the at least one facial signature algorithm comparing the at least one set of facial separation data and the plurality of sets of reference facial separation data to identify correlations between the at least one set of facial separation data and the plurality of sets of reference facial separation data; and

a network stack in communication with the at least one facial processor, the network stack being configured to transmit the at least one set of unique facial image data of each detected face to the internet protocol network from the surveillance camera.

29. A camera according to claim 28 and further comprising:

at least one compression algorithm embodied in suitable media, the at least one compression algorithm being executable with the digital format image data by at least one compressor, execution of the at least one compression algorithm producing at least one set of compressed image data;

the network stack being configured to transmit the at least one set of compressed image data to the internet protocol network.

30. A camera according to claim 29 and further comprising:

the at least one set of compressed image data including at least one set of low resolution compressed image data having a respective low resolution and at least one set of high resolution compressed image data having a respective high resolution, the low resolution being less than the high resolution.

31. A camera according to claim 30 and further comprising:

the at least one set of low resolution compressed image data including MPEG data, the at least one set of high resolution compressed image data including JPEG data.

32. A camera according to claim 30 and further comprising:

the network stack transmitting the at least one set of low resolution compressed image data according to multicast protocol.

33. A camera according to claim 28 and further comprising:

a housing, the image collection device, the at least one facial recognition algorithm, the at least one facial processor, the at least one compression algorithm, the at least one compressor and the internet protocol network stack.

34. A camera according to claim 28 and further comprising:

the network stack transmitting a portion of the at least one set of compressed image data according to multicast protocol.