Target object recognition in images and video

• US 20080159622A1
• Filed: 12/10/2007
• Published: 07/03/2008
• Est. Priority Date: 12/08/2006
• Status: Abandoned Application

• Current Assignee: The Nexus Holdings Group, LLC
• Original Assignee: The Nexus Holdings Group, LLC
• Inventors: Schieffelin, David, Borden, Walter, Agnihotri, Naveen
• Application Number:
  US12/000,153
• Publication Number:
  US 20080159622A1
• Time in Patent Office: Days
• US Class Current: 382/157
• CPC Class Codes: G06F 18/213 Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A computer-readable medium for performing target object recognition in images and video includes instructions for receiving target image data including a target object, applying non-negative matrix factorization with enforced sparseness to the target image data to generate target extracted image feature data, training a neural network to identify the target object using the target extracted image feature data to obtain a trained neural network, receiving object image data, applying non-negative matrix factorization with enforced sparseness to the object image data to generate object extracted image feature data, analyzing the object extracted image feature data with the trained neural network to obtain a result indicating whether the presence of the target object is identified in the object image data, and storing the result of analyzing the object extracted image feature data.

Claims

1. A computer-readable medium comprising instructions, which when executed by a computer system causes the computer system to perform operations for target object recognition in images and video comprising:

instructions for receiving an item of target image data, wherein the item of target image data includes a target object;

instructions for applying non-negative matrix factorization with enforced sparseness to the item of target image data to generate an item of target extracted image feature data;

instructions for training a neural network to identify the target object using the item of target extracted image feature data to obtain a trained neural network;

instructions for receiving an item of object image data;

instructions for applying non-negative matrix factorization with enforced sparseness to the item of object image data to generate an item of object extracted image feature data;

instructions for analyzing the item of object extracted image feature data with the trained neural network to obtain a result indicating whether the presence of the target object is identified in the item of object image data; and

instructions for storing the result of analyzing the item of object extracted image feature data.

2. The computer-readable medium of claim 1, further comprising instructions for processing the item of target image data with a codex before applying non-negative matrix factorization to the item of target image data.

3. The computer-readable medium of claim 1, further comprising instructions for processing an item of object image data with a codex before applying non-negative matrix factorization to the at least on item of object image data.

4. The computer-readable medium of claim 1, wherein the item of object image data is received from a webcrawler.

5. The computer-readable medium of claim 1, wherein the item of target image data is received via the Internet.

6. The computer-readable medium of claim 1, further comprising instructions for adjusting the trained neural network based on a correspondence between a node or weight in the trained neural network and a feature of the target object.

7. The computer-readable medium of claim 1, wherein the instructions for applying non-negative matrix factorization with enforced sparseness comprise:

instructions for performing factorization of a n×

m matrix V including an item of object image data or an item of target image data into non-negative matrices W and H, according to the iterative equations;

$W_{\mathrm{ia}}\leftarrow <br><br>W_{\mathrm{ia}}<br><br>\underset{\mathrm{\mu <br><br>}}{\sum <br><br>}<br><br>\frac{V_{i<br><br><br><br>\mathrm{\mu <br><br>}}}{{\left(\mathrm{WH}\right)}_{i<br><br><br><br>\mathrm{\mu <br><br>}}}<br><br>H_{a<br><br><br><br>\mathrm{\mu <br><br>}}$ $W_{\mathrm{ia}}\leftarrow <br><br>\frac{W_{\mathrm{ia}}}{\underset{j}{\sum <br><br>}<br><br>W_{\mathrm{ja}}}$ $H_{a<br><br><br><br>\mathrm{\mu <br><br>}}\leftarrow <br><br>H_{a<br><br><br><br>\mathrm{\mu <br><br>}}<br><br>\underset{j}{\sum <br><br>}<br><br>W_{\mathrm{ia}}<br><br>\frac{V_{i<br><br><br><br>\mathrm{\mu <br><br>}}}{\left(\mathrm{WH}\right)<br><br>i<br><br><br><br>\mathrm{\mu <br><br>}}$

wherein W is an i×

α

matrix, H is an α

×

μ

matrix, j is an iterator, and sparseness may be constrained at the end of every iteration according to the equations;

sparseness(w_i)=S_w,∀

i

sparseness(h_i)=S_h,‡

i

wherein w_i is the i th column of, h_i is the i th column of H, and S_w and S_h are the desired sparseness of W and H.

8. The computer-readable medium of claim 7, wherein sparseness is measured using the equation:

$\mathrm{sparseness}<br><br>\left(x\right)=\frac{\sqrt{n}-\left(\sum <br><br><br><br>x_i<br><br>\right)/\sqrt{\sum <br><br>x_i^2}}{\sqrt{n}-1}$

wherein n is the dimensionality of x.

9. The computer-readable medium of claim 1, wherein the target object is copyrighted.

10. A computer-implemented method for automated image and object recognition comprising

receiving an item of target extracted image feature data generated by applying non-negative matrix factorization with enforced sparseness to an item of object image data, wherein the item of target extracted image feature data includes a target object;

training a neural network to identify the target object using the item of target extracted image feature data to obtain a trained neural network;

receiving an item of object extracted image feature data generated by applying non-negative matrix factorization with enforced sparseness to an item of object image data;

analyzing the item of object extracted image feature data with the trained neural network to obtain a result indicating whether the presence of the target object is identified in the item of object image data; and

storing the result of analyzing the item of object extracted image feature data.

11. The computer-implemented method of claim 10, further comprising adjusting the trained neural network based on a correspondence between a one node or weight in the trained neural network and a feature of the target object.

12. The computer-implemented method of claim 10, wherein applying non-negative matrix factorization with enforced sparseness comprises:

performing factorization of a n×

m matrix V including an item of object image data or an item of target image data into non-negative matrices W and H, according to the iterative equations;

$W_{\mathrm{ia}}\leftarrow <br><br>W_{\mathrm{ia}}<br><br>\underset{\mathrm{\mu <br><br>}}{\sum <br><br>}<br><br>\frac{V_{i<br><br><br><br>\mathrm{\mu <br><br>}}}{{\left(\mathrm{WH}\right)}_{i<br><br><br><br>\mathrm{\mu <br><br>}}}<br><br>H_{a<br><br><br><br>\mathrm{\mu <br><br>}}$ $W_{\mathrm{ia}}\leftarrow <br><br>\frac{W_{\mathrm{ia}}}{\underset{j}{\sum <br><br>}<br><br>W_{\mathrm{ja}}}$ $H_{a<br><br><br><br>\mathrm{\mu <br><br>}}\leftarrow <br><br>H_{a<br><br><br><br>\mathrm{\mu <br><br>}}<br><br>\underset{j}{\sum <br><br>}<br><br>W_{\mathrm{ia}}<br><br>\frac{V_{i<br><br><br><br>\mathrm{\mu <br><br>}}}{\left(\mathrm{WH}\right)<br><br>i<br><br><br><br>\mathrm{\mu <br><br>}}$

wherein W is an i×

α

matrix, H is an α

×

μ

matrix, j is an iterator, and sparseness may be constrained at the end of every iteration according to the equations;

sparseness(w_i)=S_w,∀

i

sparseness(h_i)=S_h,‡

i

wherein w_i is the i th column of, h_i is the i th column of H, and S_w and S_h are the desired sparseness of W and H.

13. The computer-implemented method of claim 12, wherein sparseness is measured using the equation:

$\mathrm{sparseness}<br><br>\left(x\right)=\frac{\sqrt{n}-\left(\sum <br><br><br><br>x_i<br><br>\right)/\sqrt{\sum <br><br>x_i^2}}{\sqrt{n}-1}$

wherein n is the dimensionality of x.

14. An apparatus for automated image and object recognition comprising

means for receiving an item of target image data, wherein the item of target image data includes a target object;

means for applying non-negative matrix factorization with enforced sparseness to the item of target image data to generate an item of target extracted image feature data;

means for training a neural network to identify the target object using the target extracted image feature data to obtain a trained neural network;

means for receiving an item of object extracted image feature data generated by means for applying non-negative matrix factorization with enforced sparseness to an item of object image data;

means for analyzing the item of object extracted image feature data with the trained neural network to obtain a result indicating whether the presence of the target object is identified in the item of object image data; and

means for storing the result of analyzing the item of object extracted image feature data.

15. The apparatus of claim 14, further comprising means for adjusting the trained neural network based on a correspondence between a one node or weight in the trained neural network and a feature of the target object.

16. The apparatus of claim 14, wherein mean for applying non-negative matrix factorization with enforced sparseness comprises:

means for performing factorization of a n×

m matrix V including an item of object image data or an item of target image data into non-negative matrices Wand H, according to the iterative equations;

$W_{\mathrm{ia}}\leftarrow <br><br>W_{\mathrm{ia}}<br><br>\underset{\mathrm{\mu <br><br>}}{\sum <br><br>}<br><br>\frac{V_{i<br><br><br><br>\mathrm{\mu <br><br>}}}{{\left(\mathrm{WH}\right)}_{i<br><br><br><br>\mathrm{\mu <br><br>}}}<br><br>H_{a<br><br><br><br>\mathrm{\mu <br><br>}}$ $W_{\mathrm{ia}}\leftarrow <br><br>\frac{W_{\mathrm{ia}}}{\underset{j}{\sum <br><br>}<br><br>W_{\mathrm{ja}}}$ $H_{a<br><br><br><br>\mathrm{\mu <br><br>}}\leftarrow <br><br>H_{a<br><br><br><br>\mathrm{\mu <br><br>}}<br><br>\underset{j}{\sum <br><br>}<br><br>W_{\mathrm{ia}}<br><br>\frac{V_{i<br><br><br><br>\mathrm{\mu <br><br>}}}{\left(\mathrm{WH}\right)<br><br>i<br><br><br><br>\mathrm{\mu <br><br>}}$

wherein W is an i×

α

matrix, H is an α

×

μ

matrix, j is an iterator, and sparseness may be constrained at the end of every iteration according to the equations;

sparseness(w_i)=S_w,∀

i

sparseness(h_i)=S_h,‡

i

wherein w_i is the i th column of, h_i is the i th column of H, and S_w and S_h are the desired sparseness of W and H.

17. The apparatus of claim 16, wherein sparseness is measured using the equation:

$\mathrm{sparseness}<br><br>\left(x\right)=\frac{\sqrt{n}-\left(\sum <br><br><br><br>x_i<br><br>\right)/\sqrt{\sum <br><br>x_i^2}}{\sqrt{n}-1}$

wherein n is the dimensionality of x.

18. A system for automated image and object recognition

a neural network module adapted to receive target extracted image feature data generated by applying non-negative matrix factorization with enforced sparseness to target image data including target extracted image feature data for a target object, be trained to identify the target object with the target extracted image feature data, receive object extracted image feature data generated by applying non-negative matrix factorization with enforced sparseness to object image data, analyze the object extracted image feature data to obtain a result indicating whether the presence of the target object is identified in the object image data, and store the result of analyzing the object extracted image feature data for the presence of the target object in the object image data.

19. The system of claim 18, further comprising:

a serial target image input device adapted to receive the target image data including the target object, and transmit the target image data to a target image feature extraction device;

a serial object image input device adapted to receive the object image data, and transmit the object image data to an object image feature extraction device;

the target image feature extraction device adapted to receive the target image data, generate the target extracted image feature data from the target image data by applying non-negative matrix factorization with enforced sparseness to the target image data, and transmit the object extracted image feature data to the neural network module; and

the object image feature extraction device adapted receive the object image data, generate the object extracted image feature data from the object image data by applying non-negative matrix factorization with enforced sparseness to the object image data, and transmit the object extracted image feature data to the neural network module.

20. The system of claim 19, further comprising an index file storage adapted to receive the object extracted image feature data from the object image feature extraction device, store the object extracted image feature data, and transmit the object extracted image feature data to the neural network module.