REAL-TIME IDENTIFICATION OF MOVING OBJECTS IN VIDEO IMAGES

• US 20190005361A1
• Filed: 06/29/2018
• Published: 01/03/2019
• Est. Priority Date: 06/30/2017
• Status: Active Grant

• Current Assignee: AI Systems, Ltd.
• Original Assignee: AI Systems, Ltd.
• Inventors: Cho, Changwoo
• Granted Patent: US 10,726,307 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/24143 Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/168 involving transform domain methods
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/759 Region-based matching
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/776 Validation; Performance evaluation
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 2201/07 Target detection

Abstract

The disclosed technology generally relates to detecting and identifying objects in digital images, and more particularly to detecting, identifying and/or tracking moving objects in video images using an artificial intelligence neural network configured for deep learning. In one aspect, a method comprises capturing a video input from a scene comprising one or more candidate moving objects using a video image-capturing device, where the video input comprises at least two temporally spaced image frames captured from the scene. The method additionally includes transforming the video input into one or more image pattern layers, where each of the image pattern layers comprises a pattern representing one of the candidate moving objects. The method additionally includes determining a probability of match between each of the image pattern layers and a stored image in a big data library. The method additionally includes adding one or more image pattern layers having the probability of match that exceeds a predetermined level to the big data library automatically, and outputting the probability of match to a user.

Claims

1. A method implemented on an electronic device for identifying a moving object in a video image using an artificial intelligence neural network configured for deep learning, the method comprising:

capturing a video input from a scene comprising one or more candidate moving objects using a video image-capturing device, the video input comprising at least two temporally spaced image frames captured from the scene;

transforming the video input into one or more image pattern layers, wherein each of the image pattern layers comprises a pattern representing one of the candidate moving objects;

determining a probability of match between each of the image pattern layers and a stored image in a big data library;

adding one or more image pattern layers having the probability of match that exceeds a predetermined level to the big data library automatically; and

outputting the probability of match.

2. The method of claim 1, wherein transforming into one or more image pattern layers comprises transforming into one or more of:

a black pattern layer comprising a solid-filled pattern representing the one or the one or more candidate moving objects;

an outline pattern layer comprising a contiguous outer outline representing the one or more candidate moving objects; and

an inner pattern layer comprising patterns inside an outer outline representing the one or more candidate moving objects.

3. The method of claim 2, wherein transforming into one or more image pattern layers comprises reducing the size of the video input by at least three orders of magnitude.

4. The method of claim 3, wherein each of the image pattern layers and the stored image in the big data library do not exceed about 1 kilobytes and is formed of a plurality of 1 bit pixels.

5. The method of claim 3, wherein the method is performed in a single integrated device without communicating with an external processor, an external memory device or an external storage device.

6. The method of claim 3, wherein determining the probability of match comprises calculating a degree of overlap between each of the image pattern layers and a stored image in the big data library.

7. The method of claim 3, wherein the method is a deep learning method in which each of transforming the video input, determining the probability of match, and adding one or more image pattern layers to the big data library is performed automatically without intervention by a user.

8. An electronic device configured to identify a moving object in a video image using an artificial intelligence neural network configured for deep learning, comprising:

a video capturing module configured to capture a video input from a scene comprising one or more candidate moving objects using a video image-capturing device, the video input comprising at least two temporally spaced image frames captured from the scene;

an embedded software comprising;

a transformation module configured to transform the video input into one or more image pattern layers, wherein each of the image pattern layers comprises a pattern representing one of the candidate moving objects, and a matching probability determination module configured to determine a probability of match between each of the image pattern layers and a stored image in a big data library;

a big data library comprising a plurality of stored images, wherein the embedded software is configured to add one or more image pattern layers having the probability of match that exceeds a predetermined level to the big data library automatically; and

an output device configured to output the probability of match.

9. The electronic device of claim 8, wherein the transforming module is configured to transform into one or more image pattern layers comprises transforming into one or more of:

a black pattern layer comprising a solid-filled pattern representing the one or the one or more candidate moving objects;

an outline pattern layer comprising a contiguous outer outline representing the one or more candidate moving objects; and

an inner pattern layer comprising patterns inside an outer outline representing the one or more candidate moving objects.

10. The electronic device of claim 9, wherein the transforming module is configured to reduce the size of the video input by at least three orders of magnitude.

11. The electronic device of claim 9, wherein the electronic device is configured to identify the moving object as a single integrated device without communicating with an external processor, an external memory device or an external storage device.

12. The electronic device of claim 9, wherein the matching probability determination module is configured to calculate a degree of overlap between each of the image pattern layers and a stored image in the big data library.

13. The electronic device of claim 9, wherein the artificial intelligence neural network is configured for deep learning in which each of transforming the video input, determining the probability of match, and adding one or more image pattern layers to the big data library is performed automatically without intervention by a user.

14. A computer-readable medium comprising instructions that when executed cause a processor to perform the following steps for identifying a moving object in a video image using an artificial intelligence neural network configured for deep learning:

capturing a video input from a scene comprising one or more candidate moving objects using a video image-capturing device, the video input comprising at least two temporally spaced image frames captured from the scene;

transforming the video input into one or more image pattern layers, wherein each of the image pattern layers comprises a pattern representing one of the candidate moving objects;

determining a probability of match between each of the image pattern layers and a stored image in a big data library;

adding one or more image pattern layers having the probability of match that exceeds a predetermined level to the big data library automatically; and

outputting the probability of match.

15. The computer-readable medium of claim 14, wherein transforming into one or more image pattern layers comprises transforming into one or more of:

a black pattern layer comprising a solid-filled pattern representing the one or the one or more candidate moving objects;

an outline pattern layer comprising a contiguous outer outline representing the one or more candidate moving objects; and

an inner pattern layer comprising patterns inside an outer outline representing the one or more candidate moving objects.

16. The computer-readable medium of claim 15, wherein transforming into one or more image pattern layers comprises reducing the size of the video input by at least three orders of magnitude.

17. The computer-readable medium of claim 16, wherein each of the image pattern layers and the stored image in the big data library do not exceed about 1 kilobytes and is formed of a plurality of 1 bit pixels.

18. The computer-readable medium of claim 16, wherein the method is performed in a single integrated device without communicating with an external processor, an external memory device or an external storage device.

19. The computer-readable medium of claim 16, wherein determining the probability of match comprises calculating a degree of overlap between each of the image pattern layers and a stored image in the big data library.

20. The computer-readable medium of claim 16, wherein the artificial intelligence neural network is configured for deep learning in which each of transforming the video input, determining the probability of match, and adding one or more image pattern layers to the big data library is performed automatically without intervention by a user.

21. A method of identifying a moving object in one or more image frames of a video image using an artificial intelligence neural network configured for deep learning, the method comprising:

receiving an input image data from a video image-capturing device, the input image data comprising at least two image frames captured from a scene;

identifying one or more candidate moving objects in the scene based on whether a change in RGB values has occurred in pixels associated with the one or more candidate moving objects between two of the at least two image frames;

creating a plurality of image layers corresponding to each of the at least two image frames having the one or more identified candidate moving objects, wherein each of the image layers comprises an extracted image portion which includes the pixels associated with the one or more candidate moving objects;

creating a set of noise-filtered image layers from the plurality of image layers, wherein each noise-filtered image layer of the set of noise-filtered image layers includes one or more moving candidate having greater than a predetermined minimum number of pixels;

transforming the set of noise-filtered image layers into a set of gray scale image layers;

transforming each image layer of the set of gray scale image layers into one or more image pattern layers, wherein each of the one or more image pattern layers comprises a pattern representing one of the one or more candidate moving objects;

determining a probability of match between pixels of each of the one or more image pattern layers and pixels of a stored image, wherein the stored image is one of a plurality of stored images in a big data library; and

adding one or more image pattern layers having the probability of match that exceeds a predetermined level of match to the big data library; and

outputting the probability of match to a user.

22. The method of claim 21, wherein transforming into one or more image pattern layers comprises transforming into one or more of:

a black pattern layer comprising a solid-filled pattern representing the one or the one or more candidate moving objects;

an outline pattern layer comprising a contiguous outer outline representing the one or more candidate moving objects; and

an inner pattern layer comprising patterns inside an outer outline representing the one or more candidate moving objects.

23. The method of claim 21, wherein the method is a deep learning method in which creating the plurality of image layers, creating the set of noise-filtered image layers, transforming a set of noise-filtered image layers into the set of gray scale image layers, transforming each image layer of the set of gray scale image layers into one or more image pattern layers, determining a probability of match, and adding one or more image pattern layers to the big data library is performed automatically without intervention by the user.