SYSTEM AND METHOD FOR PARALLEL IMAGE PROCESSING

• US 20080123967A1
• Filed: 11/08/2007
• Published: 05/29/2008
• Est. Priority Date: 11/08/2006
• Status: Active Grant

• Current Assignee: Nextgenid, Inc. (Zeva, Inc.)
• Original Assignee: CryptoMetrics, Inc.
• Inventors: Berini, Dario, Van Beek, Gary, Fevens, Bryon, Srivastava, Prateek, Dadrassan, Hooman, Moica, Simion Adrian
• Granted Patent: US 8,295,649 B2
• Time in Patent Office: Days
• US Class Current: 382/218
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition
• CPC Class Codes: G08B 13/1961 Movement detection not involving frame subtraction, e.g. motion detection on the basis of luminance changes in the image
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet

Abstract

A system and method for processing images includes a plurality of image providers configured to transmit images. A plurality of destination processors receives the transmitted images and transforms the transmitted images to internally useable image data. A plurality of feature object engines find and identify in the internally useable image data a plurality of objects. A plurality of object classifier engines index and classify the plurality of objects found by the feature object engines.

Claims

1. An autonomous system for processing video streams and/or single images, comprising:

a plurality of image providers configured to transmit images;

a plurality of destination processors configured to receive the transmitted images and to transform the transmitted images to internally useable image data;

a plurality of feature object engines configured to find and identify in the internally useable image data a plurality of objects; and

a plurality of object classifier engines configured to index and classify the plurality of objects found by the feature object engines.

2. The system for processing images of claim 1, further comprising a plurality of metadata inserters configured to embed static and/or dynamic information into the image transmission stream.

3. The system for processing images of claim 2, further comprising a plurality of decision engines that store and/or generate actionable activities based on the output of the destination processors and the metadata provided by the metadata inserters.

4. The system for processing images of claim 1, further comprising a plurality of intelligent video multiplexer/routers that use the metadata embedded in the video stream to route the video data to one more downstream destination processors.

5. The system for processing images of claim 1, wherein the plurality of image providers includes at least one image provider selected from the list consisting of:

an analog or digital video camera, an analog or digital video recorder, a static image acquisition device, or a static image storage device.

6. The system for processing images of claim 1, wherein the destination processors include at least one image provider selected from the list consisting of:

a computer-based client application, a visual transducer, a video transducer, or an audio transducer.

7. The system for processing images of claim 2, wherein the static and/or dynamic information includes static data selected from the list consisting of:

name of the camera location, serial number of the camera, image source or identification file.

8. The system for processing images of claim 2, wherein the static and/or dynamic information includes dynamic data related to an environmental monitoring system.

9. The system for processing images of claim 8, wherein the environmental monitoring system comprises at least one system selected from the set consisting of:

fire alarms in the surveillance area, smoke alarms in the surveillance area, noise detector in surveillance area, motion detector in the surveillance area, current physical location of the camera using devices such as GPS system or beacons, current compass bearing where the camera is pointing, timestamp, auxiliary alarm system, lojack in a stolen car, access and proximity sensors, or control systems.

10. The system for processing images of claim 2, wherein the static and/or dynamic information includes dynamic data generated at the request of a person selected from the list consisting of:

officer in distress, person at the front door, first responders, panic buttons, voice commands and recognition systems.

11. The system for processing images of claim 4, wherein the intelligent video multiplexer/routers use the metadata to dynamically prioritize the video streams to ensure that higher priority streams are assigned to operational downstream destination processors before assigning the lower priority streams.

12. The system for processing images of claim 11, wherein the system is configured such that the lower priority stream is dropped in the case of downstream destination process unavailability.

13. The system for processing images of claim 4, wherein the intelligent video multiplexer/routers use the static and/or dynamic information to assign a video stream to more than one downstream destination processor to allow simultaneous identification of two or more different kinds of objects in the video stream.

14. The system for processing images of claim 4, wherein the intelligent video multiplexer/routers use the static and/or dynamic information to assign more than one video streams to one downstream destination processor to maximize resource usage.

15. The system for processing images of claim 1, wherein the system is configured to utilize destination processors based on one or more of the following:

application specific hardware architecture, reconfigurable hardware architecture, or embedded processors architecture.

16. The system for processing images of claim 15, wherein the system is configured to utilize a flexible and scalable modular deployment configuration tailored to an application'"'"'s need where destination processors are added as required to deploy additional functionality.

17. The system for processing images of claim 15, wherein destination processors are configured to alter their behavior, and be targeted to find a different class of object.

18. The system for processing images of claim 1, further comprising one or more decision engines which generate alarms in response to the classification of the objects found by one of the feature object engines.

19. The system for processing images of claim 18, wherein the alarm is accompanied by metadata related to the triggering event.

20. The system for processing images of claim 19, wherein the static and/or dynamic information includes data related to the target object, selected from the list consisting of:

position of the object, orientation of the object, position and orientation of sub-feature of the object, quality of the image, blur factor, or brightness/contrast factors.

21. The system for processing images of claim 20, wherein, in the case of a face object, the static and/or dynamic information includes data selected from the list consisting of:

the position of eyes, the position of tip of the nose or corners of mouth, orientation of the face pitch, roll, and yaw angles.

22. The system for processing images of claim 2, wherein the static and/or dynamic information that was inserted before the image processing, and the metadata generated after image processing, is used to generate an actionable activity selected from the set consisting of:

decision to send the data and image for long term storage, send image data to video displays being monitored by operators, perform object recognition, send pertinent information to field dispatch systems, or raise visual/audio alarms.

23. The system for processing images of claim 1, wherein the system is configured for face recognition and outputs a standardized format video stream.

24. The system for processing images of claim 23, wherein the standardized format video stream frames are processed and normalized to ICAO International Civil Aviation Organization static image format, type 2 and/or type 3, ISO International Organization for Standardization format or INCITS International Committee for Information Technology Standards or any other required image standards.

25. The system for processing images of claim 1, wherein at least one of the image providers is an image enhancement device comprising:

an image provider configured to transmit an image to an image enhancement device, or an image enhancement device configured to add additional information to the image and to retransmit the image.

26. The system for processing images of claim 25, wherein at least one image provider is a video camera and the additional data is embedded in the camera information on the Vertical Blanking Interval.

33. The system for processing images of claim 1, further comprising a system management engine which manages the system and performs system management functions.

34. The system for processing images of claim 33, wherein the system management engine performs one or more operations selected from the list consisting of:

status report generation, reconfiguration, code updates and code upgrades.

27. A method for processing images, comprising the steps of:

transmitting images to a plurality of destination processors;

receiving transmitted images at the plurality of destination processors;

transforming the received images to internally useable image data; and

using the plurality of destination processors to identify a plurality of objects in the internally useable image data.

28. The method of claim 27, further comprising the step of classifying the plurality of objects identified in the internally usable image data.

29. The method of claim 27, further comprising the step of generating an alarm in response to classification of the plurality of objects identified in the internally usable image data.

30. The method of claim 27, further comprising the step of enhancing at least one transmitted image with additional information.

31. The method of claim 30, wherein the image transmitted is from a video camera and the additional data is embedded in the camera information on the Vertical Blanking Interval.

32. The method of claim 31, wherein the additional information includes at least one item selected from the list consisting of:

a unique identification number, a name and location, optional location data, or compass orientation data.