SYSTEMS, METHODS AND COMPUTER PROGRAM PRODUCTS FOR IDENTIFYING OBJECTS IN VIDEO DATA

• US 20130216094A1
• Filed: 08/21/2012
• Published: 08/22/2013
• Est. Priority Date: 01/25/2012
• Status: Active Grant

• Current Assignee: Bruno Delean
• Original Assignee: Bruno Delean
• Inventors: DeLean, Bruno
• Granted Patent: US 9,336,456 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Image based operating systems and methods are provided that identify objects in video data and then take appropriate action in a wide variety of environments. In some embodiments, the image based operating systems and methods allow a user to activate other devices and systems by making a gesture.

Claims

1. A method of identifying an object in a feed of images, the method comprising the steps of:

a. analyzing a plurality of sequential frames from a feed of frames;

b. selecting in each frame, one or more unit areas of interest;

c. applying to each unit area a plurality of detectors, wherein each detector provides a value expressing one aspect of a target object;

d. assembling data in a sample vector, wherein the data comprises a plurality of components and each component is the individual value outcome of a given detector applied on a given unit area from a given frame; and

e. evaluating the performance of the sample vector by counting in the vicinity of the sample vector, all vectors known with the target object to form a first count, and counting all vectors known with or without the target object, as acquired during a training phase to form a second count, wherein the ratio of the first count to the second count estimates the probability of the presence of the target object.

2. The method of claim 1, wherein the target object is a global target that can be decomposed in subparts and each subpart is a component of the sample vector.

3. The method of claim 2, wherein each individual subpart has been trained on a specific set of data and wherein consolidated detection is later trained on a set involving the final global target.

4. The method of claim 1, wherein each of the plurality of components comprises an aggregate score.

5. The method of claim 1, wherein the data is in a form of a two dimensional matrix.

12. An image based operating system, comprising:

a camera; and

a computer in data connection with the camera, the computer having or being in communication with an executable computer program code for implementing the method of claim 1 and a module for outputting the results of said method, wherein if a threshold probability of presence is met said results indicate that an object has been identified.

13. The image based operating system according to claim 12, wherein after the object has been identified, the image based operating system causes another system to activate.

14. The image based operating system according to claim 13, wherein the another system comprises at least one of a security system, an alarm system, a communications system, an automated teller system, a banking system, a safe, another camera system, a speaker system, a microphone, another computer, a server, a laptop, a handheld computer, a bluetooth enabled device, an entertainment system, a television, a recorder, an appliance, a tool, an automobile system, a transportation system, a vehicle system, a sensor, an emitter, a transmitter, a transceiver, an antenna, a transponder, a gaming system, a computer network, a home network, a local area network, a wide area network, the Internet, the worldwide web, a satellite system, a cable system, a telecommunications system, a modem, a telephone, a cellular phone, or a smart phone.

15. The image based operating system according to claim 14, wherein the operating system is capable of identifying a characteristic in the candidate image and taking an action based on the characteristic.

16. The image based operating system according to claim 15, wherein the characteristic is at least one of:

movement of an item in an environment;

movement of an item into an environment;

movement of an item out of an environment;

appearance of an item in an image;

movement of a hand, finger, limb, head, neck, face, shoulder or a gait of a person;

action of a pet, action of a child, action of a person with special needs, or action of an elderly person.

17. The image based operating system according to claim 16, wherein the characteristic is a gesture.

18. The image based operating system according to claim 17, wherein the camera is located in an environment and the environment is at least one of an airport, an airplane, a transportation venue, a bus, a bus station, a train, a train station, a rental car venue, a car, a truck, a van, a workplace, a ticketed venue, a sports arena, a concert arena, a stadium, a sports venue, a concert venue, a museum, a store, a home, a pool, a gym, a health club, a golf club, a tennis club, a parking lot, an ATM, a storage location, a safe deposit box, a bank, or an office.

19. The image based operating system according to claim 18, wherein the environment is the home and wherein the system is used to control access to all or parts of the home based on matching a hand gesture.

20. The image based operating system according to claim 19, wherein the another system triggers an alarm and the object that has been identified is at least one of an intruder, smoke, fire, water, proximity of a child to a dangerous condition, or proximity of an elderly person to a dangerous condition.

21. The image based operating system according to claim 12, wherein the object that has been identified is at least one gesture comprising a thumbs up sign, a thumbs down sign, an okay sign, at least one finger, a hand motion, a head motion, a sign language motion, a lip motion, or an eye motion.

22. The image based operating system according to claim 12, wherein the object that has been identified is in close proximity to another object and is about to collide with the another object.

6. A method of identifying an object in a feed of images, the method comprising the steps of:

receiving video data, wherein said video data corresponds to a plurality of frames of sequential images from said feed of images;

identifying a set of candidate objects from within said feed of images, wherein said identifying comprises applying a recognition operation to data corresponding to a subset of images from each of at least two of said plurality of frames to generate a individual recognition score for a plurality of candidate objects for one or more of the subset of images in which a candidate object is present;

aggregating the recognition scores of a plurality of candidate objects that appear in a plurality of subsets of images across said at least to of said plurality of frames to generate an aggregate score for each object, wherein said aggregate score indicates a relative likelihood of presence of an object in the video data;

obtaining a subset of candidate objects based on said aggregate scores; and

comparing a change in aspect of at least one of said subset of candidate objects across said at least two of said plurality of frames to an established scenario of transformation of aspect, thereby determining an object probability score for each of one or more candidate objects from said subset, wherein an object probability score reflects the probability of the presence of an object in said video data.

7. The method according to claim 6, wherein said identifying further comprises filtering the video data.

8. The method according to claim 7, wherein an appearance of an object is detected based at least on a presence of a high-ranking recognition score for a candidate object from one frame combined with an absence of a high-ranking recognition score for a corresponding image from a second frame.

9. The method according to claim 6 further comprising calculating an absolute value of the difference between a first vector and a second vector, wherein the first vector defines a relative position of an object in a first frame, and a second vector defines a relative position of the object in a second frame, thereby determining if there was movement of the object.

10. The method according to claim 6, wherein the highest object probability score indicates the presence of an identified object and the method further comprises the steps of:

analyzing movement of a candidate object from said subset of candidate objects across a plurality of frames to verify that the movement conforms with a particular kinetic pattern so as to generate a verified object behavior; and

providing a signal based on the verified object behavior to a computer or other device.

11. The method according to claim 10, wherein the kinetic pattern comprises any of a defined path or set of paths, a change in size, or a change in rotation, or any combination thereof.

23. A computer readable non-transitory storage medium storing instructions that, when executed by a computer, causes the computer:

to receive video data, wherein said video data corresponds to a plurality of frames of sequential images;

to apply a recognition operation to data corresponding to a subset of images from at least two of said plurality of frames to generate a recognition score for a plurality of candidate objects for one or more of the subset of images in which a candidate object is present;

to aggregate the recognition scores of a plurality of candidate objects that appear in a plurality of subsets of images across said at least two of said plurality of frames to generate an aggregate score for each object, wherein said aggregate score indicates a relative likelihood of presence of an object in the video data;

to obtain a subset of candidate objects based on said aggregate scores; and

to compare a change in aspect of at least one of said subset of candidate objects across said at least to of said plurality of frames to an established scenario of transformation of aspect, thereby determining an object probability score for each of one or more candidate objects from said subset, wherein an object probability score reflects the probability of the presence of an object in said video data.

24. The computer readable storage medium of claim 23, wherein the instructions further cause the computer to output an object identifier, wherein the object identifier states the presence of any object if the probability presence is above a predetermined level.

25. The computer readable storage medium of claim 24, wherein the output further comprises a confidence interval.