System for image analysis in a network that is structured with multiple layers and differentially weighted neurons

• US 7,292,723 B2
• Filed: 02/25/2004
• Issued: 11/06/2007
• Est. Priority Date: 02/26/2003
• Status: Expired due to Fees

• Current Assignee: Inventor Holdings LLC (Walker Innovation Inc. (f/k/a Patent Properties, Inc.))
• Original Assignee: Walker Digital LLC (Walker Innovation Inc. (f/k/a Patent Properties, Inc.))
• Inventors: Tedesco, Daniel E., Jorasch, James A., Gelman, Geoffrey M., Walker, Jay S., Tulley, Stephen C., O'Neil, Vincent M., Alderucci, Dean P.
• Primary Examiner(s): DESIRE, GREGORY M
• Application Number:
  US10/786,831
• Publication Number:
  US 20040190767A1
• Time in Patent Office: 1,350 Days
• Field of Search: 382/103, 382/107, 382/117, 382/159, 382/118, 340/541, 348/143, 348/152, 348/153, 348/159, 702/188
• US Class Current: 382/159
• CPC Class Codes: A61P 29/00 Non-central analgesic, antipyretic or antiinflammatory agents, e.g. antirheumatic agents; Non-steroidal antiinflammatory drugs [NSAID]
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19663 Surveillance related processing done local to the camera
• CPC Class Codes: G08B 13/19673 Addition of time stamp, i.e. time metadata, to video stream
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Abstract

Disclosed herein are systems and methods for facilitating the usage of an online workforce to remotely monitor security-sensitive sites and report potential security breaches.

In some embodiments, cameras are configured to monitor critical civilian infrastructure, such as water supplies and nuclear reactors. The cameras are operatively connected to a central computer or series of computers, and images captured by the cameras are transmitted to the central computer. After initially registering with the central computer, Guardians “log on” to a central website hosted by the central computer and monitor the images, thereby earning compensation. In one embodiment, images of“no-man” zones—areas where no humans or vehicles should be present or monitored—are sent to Guardians for a simple determination of whether or not a human exists in the picture. Site owners compensate the operator of the computer system for this monitoring service, and the operator in turn compensates Guardians based on, for example, (i) the amount of time spent monitoring, and/or (ii) the degree of a given Guardian'"'"'s responsiveness to real or fabricated security breaches.

Among other things, systems and methods are disclosed herein for facilitating (a) the registration of monitored sites, (b) the registration of Guardians, and (c) the management of monitoring sessions, including the routing of real and fabricated images to Guardians, the rating of Guardians, and the compensation of Guardians. Many additional and alternative embodiments are contemplated, including the following example of one embodiment.

Claims

1. An apparatus comprising:

a plurality of image capture devices, each of which is configured to capture images and to transmit captured images; and

a node in communication with the image capture devices, in which the node is configured to;

receive the captured images;

store the captured images;

associate each captured image with a respective site location based on its respective image capture device;

select at least one neuron;

transmit a first one of the captured images to the at least one neuron;

receive a result from the at least one neuron;

determine whether the result indicates that a human is present in the first captured image;

select at least one additional neuron based on whether the result indicates that a human is present in the first captured image;

transmit the first captured image to the at least one additional neuron;

receive at least one additional result from the at least one additional neuron;

determine whether the at least one additional result indicates that a human is present in the first captured image; and

determine an alarm condition based on the at least one additional result.

2. The apparatus of claim 1, in which at least one image capture device is configured to preprocess captured images.

3. The apparatus of claim 1, in which the node is further configured to preprocess at least some of the captured images.

4. The apparatus of claim 1, in which at least one image capture device is configured to:

detect motion; and

capture an image in response to the detected motion.

5. The apparatus of claim 1, in which the result received from the at least one neuron is a one-bit indication of the result.

6. The apparatus of claim 1, in which the result received from the at least one neuron indicates one of:

the presence of a human in the first captured image,

the absence of a human in the first captured image, and

uncertainty as to whether there is a human present in the first captured image.

7. The apparatus of claim 1, in which the at least one neuron has an associated weight, and in which the node is configured to select the at least one additional neuron based further on the weight of the at least one neuron.

8. The apparatus of claim 1, in which the at least one neuron has an associated weight, and in which the node is configured to determine an alarm condition based further on the weight of the at least one neuron.

9. The apparatus of claim 1, in which each neuron has an associated weight, and in which the node is configured to adjust the weights based on the results.

10. The apparatus of claim 1, further comprising:

a layer analyzer which is configured to allocate the neurons to different layers, based on respective weights of the neurons.

11. The apparatus of claim 1, in at least some of the image capture devices are arranged to capture images from areas in which human activity is desired to be substantially nonexistent.

12. A method comprising:

receiving a captured image from each of a plurality of image capture devices;

storing the captured images;

associating each captured image with a respective site location based on its respective image capture device;

selecting at least one neuron;

transmitting a first one of the captured images to the at least one neuron;

receiving a result from the at least one neuron;

determining whether the result indicates that a human is present in the first captured image;

selecting at least one additional neuron based on whether the result indicates that a human is present in the first captured image;

transmitting the first captured image to the at least one additional neuron;

receiving at least one additional result from the at least one additional neuron;

determining whether the at least one additional result indicates that a human is present in the first captured image; and

determining an alarm condition based on the at least one additional result.

13. The method of claim 12, in which the result received from the at least one additional neuron indicates one of:

the presence of a human in the first captured image,

the absence of a human in the first captured image, and

uncertainty as to whether there is a human present in the first captured image.

14. The method of claim 12, in which each neuron has an associated weight, and in which the node is configured to adjust the weights based on the results.

15. A method, comprising:

receiving, from a first sensor associated with a first remote location, first sensed data;

storing an indication of an association between the first sensed data and the first remote location;

selecting, after the receiving, at least one neuron from a plurality of available neurons;

sending the first sensed data to the selected at least one neuron;

receiving, after the sending of the first sensed data to the selected at least one neuron, a result from the at least one neuron;

determining that the result is indicative of a detection of a human presence in association with the first sensed data;

selecting, after the determining that the result is indicative of a detection of a human presence in association with the first sensed data, at least one additional neuron from the plurality of available neurons;

sending the first sensed data to the selected at least one additional neuron;

receiving, after the sending of the first sensed data to the selected at least one additional neuron, an additional result from the at least one additional neuron;

determining that the additional result is also indicative of a detection of a human presence in association with the first sensed data; and

determining, based at least in part on the determining that the additional result is also indicative of a detection of a human presence in association with the first sensed data, an alarm condition.

16. The method of claim 15, wherein the first sensed data is received pursuant to a transmission of an indication of the first sensed data from the first sensor, and wherein the transmission is triggered by a detection of motion at the first remote location.

17. The method of claim 15, wherein the first sensor pre-processes the first sensed data by utilizing pre-processing software.

18. The method of claim 17, wherein the pre-processing software is configured based on a characteristic of the first remote location.

19. The method of claim 15, further comprising:

incorporating one or more of a digital watermark and a digital fingerprint into the first sensed data.

20. The method of claim 15, wherein selecting of the at least one neuron comprises:

determining a type of data associated with the first sensed data; and

selecting the at least one neuron from the plurality of available neurons based on a weight of the at least one neuron that is associated with the same type of data of the first sensed data.

21. The method of claim 20, wherein the type of data comprises one or more of (i) a type of image, (ii) a captured image, and (iii) audio data.

22. The method of claim 15, wherein the first sensed data comprises a first captured image.

23. The method of claim 15, wherein the first sensed data comprises a first sensed audio.

24. A method, comprising:

receiving an image from an image capture device, wherein the image is of an area in which human activity is desired to be substantially nonexistent;

receiving, after the receiving of the image from the image capture device, a first monitoring session initiation request from a first user and a second monitoring session initiation request from a second user;

receiving a first user identifier from the first user and a second user identifier from the second user;

verifying that the first user identifier corresponds to the first user and that the second user identifier corresponds to the second user;

providing, after the verifying, the image to the first and second users;

receiving, after the providing, a first response associated with the image from the first user and a second response associated with the image from a second user, wherein the first and second responses comprise indications that a human is present in the image;

determining an entity associated with the area; and

notifying the entity that a security breach may exist at the area.

25. A method, comprising:

registering a plurality of sites, wherein the registration of each site comprises receiving a request from an entity associated with one of the plurality of sites, the request comprising an indication that the entity desires to register the one of the plurality of sites to be monitored by a plurality of remote patrollers;

determining at least one security camera associated with each of the plurality of registered sites;

testing the plurality of security cameras;

providing, after the testing and to the plurality of remote patrollers, (i) an image captured by one of the plurality of cameras associated with the one of the plurality of sites, and (ii) a request for a binary determination associated with the image;

receiving the binary determination from each of the plurality of remote patrollers;

determining that at least one of the plurality of binary determinations indicates a positive result; and

triggering, based on the determining that the at least one of the plurality of binary determinations indicates the positive result, an alarm condition.

26. The method of claim 25, wherein the image captured by the one of the plurality of cameras associated with the one of the plurality of sites is provided to the plurality of remote patrollers as an image associated with a reel of a slot machine.

27. The method of claim 25, further comprising:

facilitating a conferring regarding the image amongst the plurality of remote patrollers.

28. The method of claim 27, wherein the facilitating comprises initiating one or more instant messaging sessions associated with the image.

29. The method of claim 25, wherein the binary determination comprises a determination of whether less than two people are present in the image.