SYSTEM AND METHOD FOR OBJECT RECOGNITION AND TRACKING IN A VIDEO STREAM

• US 20110291925A1
• Filed: 02/02/2010
• Published: 12/01/2011
• Est. Priority Date: 02/02/2009
• Status: Active Grant

• Current Assignee: eyeSight Mobile Technologies Ltd.
• Original Assignee: eyeSight Mobile Technologies Ltd.
• Inventors: Shenfeld, Amnon, Cohen, Dudi, Israel, Nadav, Katz, Itay
• Granted Patent: US 9,405,970 B2
• Time in Patent Office: Days
• US Class Current: 345/157
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Provided is a system and method for object detection and tracking in a video stream. Frames of the video stream are divided into regions of interest and a probability that the region contains at least a portion of an object to be tracked is calculated for each region of interest. The regions of interest in each frame are then classified based on the calculated probabilities. A region of interest (RI) frame is then constructed for each video frame that reports the classification of regions of interest in the video frame. Two or more RI frames are then compared in order to determine a motion of the object. Also provided is a system executing the presently described method, as well as a device including the system. The device may be for example, a portable computer, a mobile telephone, or an entertainment device.

Claims

1-30. -30. (canceled)

31. A system for object detection and tracking in a video stream, comprising:

(a) a processor comprising an object detection module and an object tracking module;

wherein the object detection module is configured to;

(i) calculate, for each of one or more regions of interest in each of two or more frames in the video stream, a probability that the region of interest contains at least a portion of an object to be tracked; and

(ii) classify the regions of interest in each of the two or more frames according to the calculated probabilities and generate a region of interest (RI) frame for each video frame, the RI frame reporting the classification of regions of interest; and

wherein the object tracking module is configured to;

(i) compare two RI frames generated by the object detection module and determine a motion of the object.

32. The system according to claim 31, wherein the object tracking module comprises one or more pattern detection modules, each pattern detection module being configured to calculate a probability that a specific pattern of motion of the tracked object during a time window occurred during the time window.

33. The system according to claim 32, wherein the object tracking module further comprises a motion recognition module determining a motion pattern most likely to have occurred based upon the probabilities generated by the one or more pattern detection modules.

34. The system according to claim 33, wherein the determination of the motion recognition module takes into account an external signal.

35. The system according to claim 31, further comprising an operating system (OS) command execution module configured to execute an OS command associated with an identified pattern of motion.

36. The system according to claim 31, wherein the probability that a region of interest contains at least a portion of the object to be tracked is obtained by a method comprising:

(a) for each of one or more regions of interest in each frame in the video stream, calculating a statistical analysis of the pixels in the region of interest; and

(b) calculating a discrete classification of the region of interest in a calculation involving statistical analysis of the region of interest in one or more previous frames of the video stream.

37. The system according to claim 36, wherein the statistical analysis comprises generating a histogram for each of one or more functions defined on pixels of the region of interest.

38. The system according to claim 37, wherein one or more of the functions are selected from the group consisting of

(a) an intensity of any one of the colors red, green, or blue of the pixels;

and

(b) any one of a hue, saturation or luminance of the pixels.

39. The system according to claim 37, further comprising calculating values of statistical parameters of one or more of the functions.

40. The system according to claim 39, wherein one or more of the statistical parameters are selected from the group consisting of

(a) an average,

(b) a mode,

(c) a standard deviation, and

(d) a variance.

41. The system according to claim 37, wherein the statistical analysis further comprises calculating a distance between one or more generated histograms and a histogram indicative of the presence of the object to be tracked.

42. The system according to claim 35, wherein the step of comparing two or more RI frames comprises:

(a) for each frame, and for each of the classified regions of interest in the frame, comparing the classification of the region of interest with the classification of the region of interest in a plurality of frames obtained in a time window containing the frame;

(b) determining, on the basis of the comparison, whether or not the selected region of interest contains the object to be tracked;

(c) reclassifying, on the basis of this determination, the region of interest, according to whether or not the region of interest contains the object to be tracked; and

(d) calculating one or more tracking parameters of the object'"'"'s motion based upon changes in the states of two or more of the regions during a time window.

43. The system according to claim 42, wherein the tracking parameters are selected from the group consisting of

(a) direction of movement of the object,

(b) a speed of movement of the object,

(c) an acceleration of the object,

(d) a width of the object in pixels,

(e) a height of the object in pixels, and

(f) location of the object in the frame.

53. A data processing device comprising a system according to claim 31.

54. The data processing device according to claim 53, selected from the group consisting of

(a) a personal computer (PC),

(b) a portable computer such as a PDA or a laptop,

(c) a mobile telephone,

(d) a radio,

(e) an entertainment device,

(f) Smart Home,

(g) a vehicle,

(h) a digital camera,

(i) kitchen appliance,

(j) an media player or media system,

(k) location based devices,

(l) a mobile game machine,

(m) Pico projector or embedded projector,

(n) Medical display device, and

(o) In-car/in-air Infotainment system.

55. The device according to claim 53, further comprising one or both of a video camera and a display screen.

56. The device according to claim 53, wherein one or more of the patterns of motion of the tracked object are selected from the group consisting of

(a) a width of the object in pixels increased during the time window,

(b) the width of the object in pixels decreased during the time window,

(c) the object moved closer to the camera,

(d) the object moved away from the camera,

(e) the object moved in a predetermined path,

(f) the object rotated,

(g) the object was stationary,

(h) the object performed any type of motion,

(i) the object performed a flicking motion,

(j) the object accelerated,

(k) the object decelerated, and

(l) the object moved and then stopped.

44. A method for object detection and tracking in a video stream, comprising:

(i) calculating, for each of one or more regions of interest in each of two or more frames in the video stream, a probability that a region of interest contains at least a portion of an object to be tracked;

(ii) classifying the regions of interest in each of the two or more frames according to the calculated probability and generating a region of interest (RI) frame for each video frame, the RI frame reporting the classification of regions of interest; and

(iii) comparing two or more RI frames generated by the object detection module and determine a motion of the object.

45. The method according to claim 44, wherein the probability that a region of interest contains at least a portion of the object to be tracked is obtained by a method comprising:

(a) for each of one or more regions of interest in each frame in the video stream, calculating a statistical analysis of the pixels in the region of interest; and

(b) calculating a discrete classification of the region of interest in a calculation involving the statistical analysis of the region of interest in one or more previous frames of the video stream.

46. The method according to claim 45, wherein the statistical analysis comprises generating a histogram for each of one or more functions defined on pixels of the region of interest.

47. The method according to claim 46, wherein one or more of the functions are selected from the group consisting of

(a) an intensity of any one of the colors red, green, or blue of the pixels, and

(b) any one of a hue, saturation or luminance of the pixels.

48. The method according to claim 46, further comprising calculating values of statistical parameters of one or more of the functions.

49. The method according to claim 48, wherein one or more of the statistical parameters are selected from the group consisting of

(a) an average,

(b) a mode,

(c) a standard deviation, and

(d) a variance.

50. The method according to claim 46, wherein the statistical analysis further comprises calculating a distance between one or more generated histograms and a histogram indicative of the presence of the object to be tracked.

51. The method according to claim 44, wherein the step of comparing two or more RI frames comprises:

(a) for each frame, and for each of the classified regions of interest in the frame, comparing the classification of the region of interest with the classification of the region of interest in a plurality of frames obtained in a time window containing the frame;

(b) determining, on the basis of the comparison, whether or not the selected region of interest contains the object to be tracked;

(c) reclassifying, on the basis of this determination, the region of interest, according to whether or not the region of interest contains the object to be tracked; and

(d) calculating one or more tracking parameters of the object'"'"'s motion based upon changes in the states of two or more of the regions during a time window.

52. The method according to claim 51, wherein the tracking parameters are selected from the group consisting of

(a) direction of movement of the object,

(b) a speed of movement of the object,

(c) an acceleration of the object,

(d) a width of the object in pixels,

(e) a height of the object in pixels, and

(f) location of the object in the frame.

57. The device according to claim 52, wherein the processor further comprises an operating system (OS) command execution module configured to execute an OS command of the device associated with an identified pattern of motion.

58. The device according to claim 57, wherein one or more of the OS commands are selected from the group consisting of

(a) depressing a virtual key displayed on a display screen of the device,

(b) moving a curser appearing on a display screen of the device to a new location on the screen,

(c) rotating a selection carousel,

(d) switching between desktops,

(e) running on the central processor unit a predetermined software application,

(f) turning off an application,

(g) turning the speakers on or off,

(h) Turning volume up/down,

(i) skipping to the next or previous track in a media player or between IPTV channels,

(j) controlling a GPS application,

(k) switching on voicemail service,

(l) navigating in photo/music album gallery,

(m) scrolling web-pages, emails, documents or maps,

(n) controlling actions in mobile games, and

(o) controlling interactive video or animated content.

59. A computer program comprising computer program code means for performing all the steps of claim 44, when the program is run on a computer.

60. A computer program as claimed in claim 59, embodied on a computer readable medium.