UNIFIED SYSTEM AND METHOD FOR ANIMAL BEHAVIOR CHARACTERIZATION WITH TRAINING CAPABILITIES

• US 20110007946A1
• Filed: 09/13/2010
• Published: 01/13/2011
• Est. Priority Date: 11/24/2000
• Status: Abandoned Application

• Current Assignee: Clever Sys Incorporated
• Original Assignee: Clever Sys Incorporated
• Inventors: LIANG, Yiqing, CRNIC, Linda, KOBLA, Vikrant, WOLF, Wayne
• Application Number:
  US12/881,154
• Publication Number:
  US 20110007946A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: A01K 1/031 Cages for laboratory animals; Cages for measuring metabolism of animals
• CPC Class Codes: A01K 29/005 Monitoring or measuring activity, e.g. detecting heat or mating
• CPC Class Codes: A61B 2503/40 Animals
• CPC Class Codes: A61B 2503/42 for laboratory research
• CPC Class Codes: A61B 5/1113 Local tracking of patients, e.g. in a hospital or private home
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/4094 Diagnosing or monitoring seizure diseases, e.g. epilepsy
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G16H 40/67 for remote operation

Abstract

In general, the present invention is directed to systems and methods for finding the position and shape of an object using video. The invention includes a system with a video camera coupled to a computer in which the computer is configured to automatically provide object segmentation and identification, object motion tracking (for moving objects), object position classification, and behavior identification. In a preferred embodiment, the present invention may use background subtraction for object identification and tracking, probabilistic approach with expectation-maximization for tracking the motion detection and object classification, and decision tree classification for behavior identification. Thus, the present invention is capable of automatically monitoring a video image to identify, track and classify the actions of various objects and the object'"'"'s movements within the image. The image may be provided in real time or from storage. The invention is particularly useful for monitoring and classifying animal behavior for testing drugs and genetic mutations, but may be used in any of a number of other surveillance applications.

Claims

1-42. -42. (canceled)

43. A user configurable video-based animal behavior analysis system, comprising a computer configured to determine features of an animal from video images and to characterize activity of said animal as one of a set of behaviors based on an analysis of changes in said features over time;

wherein the set of behaviors comprises at least one predetermined standard behavior and

wherein one or more user-defined behaviors can be added to the set of behaviors.

44. The system of claim 43 wherein the user-defined behavior is entered into the system in the form of video images of the user-defined behavior.

45. The system of claim 43 wherein the user-defined behavior is entered into the system in the form of a set of rules.

46. The system of claim 43, further comprising:

a video camera and a video digitization unit coupled to said computer for capturing said video images and converting said video images from analog to digital format.

47. The system of claim 43, further comprising:

an animal identification, segregation, and tracking module receiving said video images.

48. The system of claim 47 wherein the at least one user-defined behavior is defined by analysis of video images of the user-defined behavior by the animal identification, segregation and tracking module.

49. The system of claim 47, wherein said computer further includes a behavior identification module for characterizing activity of said animal, said behavior identification module being coupled to said animal identification, segregation, and tracking module.

50. The system of claim 49, wherein said computer further includes an animal behavior storage module for storing information about the predetermined standard behaviors, said animal behavior storage module being coupled to said behavior identification module.

51. The system of claim 50 wherein the animal behavior storage module stores information about the at least one user-defined behavior.

52. The system of claim 43, wherein said animal is a mouse.

53. The system of claim 43, wherein said animal is a rat.

54. A method of characterizing activity of an animal using computer processing of video images, comprising:

detecting an animal in said video images;

tracking activity of said animal over a plurality of said video images;

classifying the activity of said animal as one of a set of classifiers based on comparing the activity over time to sets of rules of said set of classifiers;

wherein the set of classifiers comprise at least one standard classifier and

wherein one or more user-defined classifiers can be added to the set of classifiers.

55. The method of claim 54 wherein the user-defined classifier is entered into the system in the form of video images of the user-defined classifier.

56. The method of claim 54 wherein the user-defined classifier is entered into the system in the form of a set of rules.

57. The method of claim 54 further comprising:

detecting an animal in video images of the at least one user-defined classifier;

tracking activity of the animal over a plurality of the video images of the at least one user-defined classifier;

creating user-defined set of rules based on the changes in activity of the animal over a plurality of the video images of the at least one user-defined classifier;

wherein the user-defined set of rules is used to classify the activity of the animal as the at least one user-defined classifier.

58. The method of claim 54, further including characterizing activity by comparing the activity of the animal to the set of classifiers.

59. The method of claim 58, wherein said characterizing activity of said animal comprises using statistical shape information.

60. The method of claim 58, wherein said characterizing activity of said animal comprises using contour-based shape information.

61. The method of claim 58, wherein said characterizing activity of said animal includes identifying activity of the animal as one of the set of classifiers, said set of classifiers comprising a horizontal side view classifier, a vertical classifier, a cuddled classifier, a horizontal front/back view classifier, a partially reared classifier, a stretched classifier, a hang vertical classifier, a hang cuddled classifier, an eating classifier, and a drinking classifier.

62. The method of claim 54, further including characterizing activity by aggregating said classifiers into a behavior over a range of images.

63. The method of claim 62, wherein the characterizing activity by aggregating said classifiers into a behavior over a range of images includes identifying patterns of classifiers over a sequence of images.

64. The method of claim 62, wherein the aggregating of said classifiers into a behavior over a range of images includes analyzing temporal ordering of said classifiers.

65. The method of claim 64, wherein said analyzing of temporal ordering of said classifiers is performed with the use of time-series analysis such as Hidden Markov Model (HMM).

66. The method of claim 54, wherein said classifying the activity of said animal as one of the set of classifiers includes analyzing the statistical and contour-based shape information from the image.

67. The method of claim 54, wherein said set of classifiers includes an unknown classifier, wherein said unknown classifier cannot be classified as one of the set of standard or user-defined classifiers.

68. The method of claim 54, wherein the detecting is conducted under night conditions and the images are obtained using red light.

69. The method of claim 54, wherein said detecting an animal includes detecting body parts of said animal.

70. The method of claim 69, wherein said body parts include the forelimbs.

71. A method of characterizing activity of an animal using computer processing of video images, comprising

detecting an animal in said video images;

tracking activity of said animal over a plurality of video images;

characterizing the activity based on models or rules,

wherein the models or the rules comprise at least one standard model or standard rule;

wherein one or more user-defined models or user-defined rules can be added to the models or rules.

72. The method of claim 71 wherein said characterizing said activity comprises:

using statistical information.

73. The method of claim 71 wherein said characterizing activity of the animal comprises using contour-based shape information selected from the group consisting of:

area of said animal;

centroid position of said animal;

bounding box and aspect ratio of said bounding box of said animal;

eccentricity of said animal; and

directional orientation of said animal.

74. The method of claim 71 wherein characterizing activity of said animal comprises analyzing temporal information regarding the activity of the animal selected from the group consisting of direction and magnitude of movement of the centroid, increase and decrease of the eccentricity, increase and decrease of the area, increase and decrease of the aspect ratio of a bounding box, and change in contour information.

75. A trainable video-based animal behavior analysis system comprising a computer configured to track an animal over time using video images, to extract features of the animal, and to use the extracted features for training the system and classification of the organism from one frame to another frame.

76. The trainable video-based animal behavior analysis system of claim 75, wherein at least one of the extracted features is the shape of the animal.

77. The trainable video-based animal behavior analysis system of claim 75, wherein classifiers are stored in memory in the computer for later use in analyzing the behavior of the animal.

78. A trainable animal characterization system comprising a computer having an animal identification module for identifying an animal in an image, an animal tracking module for tracking the animal from one image to another, and an animal classification module for classifying the animal based upon classifiers stored in a memory of the system, wherein the classification module is configured to extract one or more features of the animal for training the classification module to identify new classifiers that are not stored in the memory, the classification module further configured to store in memory the new classifiers for use in subsequent animal characterizations.

79. The trainable animal behavior analysis system of claim 78, wherein the stored classifiers and the new classifiers are stored in a classifier module.