System and method for animal gait characterization from bottom view using video analysis

• US 20070229522A1
• Filed: 09/30/2004
• Published: 10/04/2007
• Est. Priority Date: 11/24/2000
• Status: Active Grant

• Current Assignee: Clever Sys Incorporated
• Original Assignee: Clever Sys Incorporated
• Inventors: Zhang, Yi, Liang, Yiqing, Kobla, Vikrant, Wang, Feng-Feng
• Granted Patent: US 8,514,236 B2
• Time in Patent Office: Days
• US Class Current: 345/547
• CPC Class Codes: A01K 1/031 Cages for laboratory animals; Cages for measuring metabolism of animals
• CPC Class Codes: A01K 29/005 Monitoring or measuring activity, e.g. detecting heat or mating
• CPC Class Codes: A61B 2503/40 Animals
• CPC Class Codes: A61B 2503/42 for laboratory research
• CPC Class Codes: A61B 5/1113 Local tracking of patients, e.g. in a hospital or private home
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/4094 Diagnosing or monitoring seizure diseases, e.g. epilepsy
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G16H 40/67 for remote operation

Abstract

In general, the present invention is directed to systems and methods for finding the gait parameters of an animal using video. The invention includes a system with a video camera coupled to a computer in which the computer is configured to automatically provide animal segmentation, foot identification, foot tracking, and gait parameter calculation. In a preferred embodiment, the present invention may use a treadmill apparatus with a transparent tread or belt for gait parameter calculation. A camera may be placed directly underneath the transparent tread such that the bottom view of the animal walking or running on the transparent tread is captured by the camera. A tilted mirror may also be placed directly underneath the transparent tread in order to reflect the bottom view images to the side where they are captured by a high-speed camera. Thus, the present invention is capable of automatically monitoring a video image to identify, track and classify the actions of various animals'"'"' feet within the image. The image may be provided in real time or from storage. The invention is particularly useful for monitoring and characterizing gait behavior such as walking and running behavior and their patterns for assessing neurological functions, testing drugs and genetic mutations, but may be used in any of a number of other monitoring and surveillance applications.

Claims

1. A video-based animal gait analysis system, comprising:

a computer configured to determine a position and shape of an animal and its feet from video images and characterize activity of said feet of the animal based on analysis of changes in said position and said shape over time.

2. The system of claim 1, further comprising:

a video camera and a video digitization unit coupled to said computer for capturing said video images and converting said video images from analog to digital format.

3. The system of claim 2, further comprising:

an animal identification and segregation module receiving said video images.

4. The system of claim 3, wherein said computer further includes a foot identification module for segregating said feet from the said animal, said foot identification module being coupled to said animal identification and segregation module.

5. The system of claim 4, wherein said computer further includes a foot tracking module for tracking each of the 4 feet independently, said foot tracking module being coupled to said foot identification module.

6. The system of claim 5, wherein said computer further includes a gait characterization module for extracting a plurality of gait parameters from the tracked feet, said gait characterization module being coupled to the foot tracking module.

7. The system of claim 1, wherein said animal is a mouse.

8. The system of claim 1, wherein said animal is a rat.

9. A method of determining and characterizing gait activity of an animal using computer processing of video images, comprising the steps of:

detecting an animal in said video images;

identifying all four feet of the detected animal over a plurality of said video images;

tracking each of the four feet independently; and

characterizing said tracked data into gait activity of said animal based on calculation of gait parameters.

10. The method of claim 9, wherein said step of characterizing said gait activity based on calculation of gait parameters further includes the steps of:

calculating the stride length and stride time;

calculating the stance and swing times;

calculating the brake and propulsion times of the stance;

calculating the average and instantaneous running speeds;

calculating the foot spacing information; and

calculating the coupling parameters of the feet.

11. The method of claim 9, wherein said detecting an animal includes using a background subtraction method comprising the steps of:

apply a threshold on a difference between a current image and a background so as to determine a foreground object;

refine contours of said foreground object image by smoothing to obtain the final animal object.

12. The method of claim 9, wherein said identifying all four feet of the detected animal includes using absolute and normalized natural color of the feet to model the feet and using these color models to identify the feet from the rest of the animal body.

13. The method of claim 12, wherein said step of identifying all four feet of the detected animal further includes using statistical shape and relative position information of the four detected feet to assign correspondences for each of the detected feet in the current video image to the instances of the feet from the previous video images.

14. The method of claim 9, wherein said step of tracking individual feet includes following successive correspondences from one video image to next and maintaining the list of tracked position and shape data over the course of the sequence of video images.

15. The method of claim 9, wherein said steps are performed in all types of light conditions including visible, colored and infra-red light;

16. The method of claim 9, wherein said steps are also performed with a plurality of cages or arenas, each of which contains a single animal;

17. The method of claim 9, wherein said steps are also performed with high-speed cameras with speeds of up to 200 frames per second.

18. The method of claim 9, wherein said step of detecting animal includes the step of detecting body parts of the animal;

19. The method of claim 18, wherein said body parts include the head;

20. The method of claim 18, wherein said body parts include the tail;

21. The method of claim 18, wherein said body parts include the feet;

22. The method of claim 9, wherein said video images include various animal behavioral gait analysis apparatuses.

23. The method of claim 22, wherein said various animal behavioral analysis apparatuses include treadmills with a transparent tread or belt.

24. The method of claim 23, wherein said various animal behavioral analysis apparatuses include a high-speed camera placed underneath the transparent tread or belt so that the bottom view of the animal walking or running on the said transparent tread or belt is visible.

25. The method of claim 23, wherein said various animal behavioral analysis apparatuses include a tilted mirror placed underneath the transparent tread or belt so that the bottom view of the animal walking or running on the said transparent tread or belt is reflected to the side where a high-speed camera is placed to capture the video images.

26. The method of claim 10, wherein said calculation of stride length includes calculating the distance between the initiation of contact with the tread and release of contact with the tread between which the foot remains in continuous contact with the tread.

27. The method of claim 10, wherein said calculation of stride time includes the time elapsed between two successive initiations of contact.

28. The method of claim 10, wherein said calculation of stance time includes the time elapsed between the initiation and release of contact.

29. The method of claim 10, wherein said calculation of swing time includes the time elapsed between the release of contact of one stride and initiation of contact of the successive stride.

30. The method of claim 10, wherein said calculation of brake time includes the time elapsed between the initiation of contact of one stride and instance that the area of the foot in contact with the tread reaches maximum area.

31. The method of claim 10, wherein said calculation of propulsion time includes the time elapsed between the instance that the area of the foot in contact with the tread reaches maximum area during a stance and the release of contact with the tread.

32. The method of claim 10, wherein said calculation of instantaneous running speed includes the ratio of the stride length and the stance time, and the calculation of average running speed includes the average of the instantaneous running speed over the entire sequence of video images.

33. The method of claim 10, wherein said calculation of spacing information between feet includes the distance between midpoints of the strides of following four pairs of feet:

front left foot and front right foot;

rear left foot and rear right foot;

front left foot and rear left foot; and

front right foot and rear right foot.

34. The method of claim 10, wherein said calculation of coupling measures between feet includes the relative offset expressed as a percentage, of the initiation of contact of the other three feet with respect to the stride of the reference foot.