System and method for animal gait characterization from bottom view using video analysis

• US 8,514,236 B2
• Filed: 09/30/2004
• Issued: 08/20/2013
• Est. Priority Date: 11/24/2000
• Status: Active Grant

• Current Assignee: Clever Sys Incorporated
• Original Assignee: Clever Sys Incorporated
• Inventors: Kobla, Vikrant, Liang, Yiqing, Wang, Feng-Feng, Zhang, Yi
• Primary Examiner(s): HOLDER, ANNER N
• Application Number:
  US10/955,661
• Publication Number:
  US 20070229522A1
• Time in Patent Office: 3,246 Days
• Field of Search: 345/547
• US Class Current: 345/547
• CPC Class Codes: A01K 1/031 Cages for laboratory animals; Cages for measuring metabolism of animals
• CPC Class Codes: A01K 29/005 Monitoring or measuring activity, e.g. detecting heat or mating
• CPC Class Codes: A61B 2503/40 Animals
• CPC Class Codes: A61B 2503/42 for laboratory research
• CPC Class Codes: A61B 5/1113 Local tracking of patients, e.g. in a hospital or private home
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/4094 Diagnosing or monitoring seizure diseases, e.g. epilepsy
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G16H 40/67 for remote operation

Abstract

In general, the present invention is directed to systems and methods for finding the gait parameters of an animal using video. The invention includes a system with a video camera coupled to a computer in which the computer is configured to automatically provide animal segmentation, foot identification, foot tracking, and gait parameter calculation. In a preferred embodiment, the present invention may use a treadmill apparatus with a transparent tread or belt for gait parameter calculation. A camera may be placed directly underneath the transparent tread such that the bottom view of the animal walking or running on the transparent tread is captured by the camera. A tilted mirror may also be placed directly underneath the transparent tread in order to reflect the bottom view images to the side where they are captured by a high-speed camera. Thus, the present invention is capable of automatically monitoring a video image to identify, track and classify the actions of various animals'"'"' feet within the image. The image may be provided in real time or from storage. The invention is particularly useful for monitoring and characterizing gait behavior such as walking and running behavior and their patterns for assessing neurological functions, testing drugs and genetic mutations, but may be used in any of a number of other monitoring and surveillance applications.

Claims

1. An animal gait analysis system, comprising:

a computer configured

to determine a shape of at least one animal and position and shape of the animal'"'"'s feet from a plurality of images whether or not the feet are in contact with the floor, wherein the images are images of a bottom view of the at least one animal through a transparent floor of an arena; and

to characterize activity of the animal at least in part based on analysis of the shape of the animal and the position and shape of the feet.

2. The system of claim 1, further comprising:

a video camera for capturing the images and a video digitization unit coupled to said computer for converting said images from analog to digital format if the images are captured in analog format.

3. The system of claim 1, further comprising:

an animal identification and segregation module receiving said images and identifying and segregating the animal.

4. The system of claim 3, wherein said computer further comprises a foot identification module for segregating said feet from said animal.

5. The system of claim 4, wherein said computer further comprises a foot tracking module for tracking each foot independently.

6. The system of claim 5, wherein said computer further comprises a gait characterization module for extracting gait parameters from the tracked feet.

7. The system of claim 1, wherein said animal is a mouse.

8. The system of claim 1, wherein said animal is a rat.

22. The system of claim 1 further comprising at least one animal behavioral gait analysis apparatus.

23. The system of claim 22, wherein said animal behavioral analysis apparatus is a movable belt apparatus with a transparent belt.

24. The system of claim 23, wherein the images are captured using a high-speed camera placed below the transparent belt so that the bottom view of the animal on the transparent belt is visible.

25. The system of claim 23, wherein said animal behavioral analysis apparatus comprises a tilted mirror placed underneath the transparent belt so that the bottom view of the animal on said transparent belt is reflected to the side where a high-speed camera is placed to capture the images.

43. The system of claim 1 wherein the arena is a treadmill apparatus including a transparent movable belt;

and wherein the transparent floor is the transparent movable belt.

9. A method of analyzing gait activity of at least one animal using computer processing of images, comprising:

acquiring a plurality of images of a bottom view of the animal through a transparent floor of an arena;

analyzing the shape of the animal in said images;

identifying all feet of the animal over a plurality of said images;

determining the motion of each foot independently whether or not the feet are in contact with the floor of the arena;

calculating gait parameters of the animal based on the motion of the feet; and

characterizing gait activity of said animal based on the gait parameters.

10. The method of claim 9, wherein calculating gait parameters of the animal further comprises at least one of the following steps:

calculating the stride length and stride time;

calculating the stance and swing times;

calculating the brake and propulsion times of the stance;

calculating the average and instantaneous running speeds;

calculating the foot spacing information; and

calculating the coupling parameters of the feet.

11. The method of claim 9, wherein said analyzing the shape of the animal comprises using a background subtraction method comprising:

applying a threshold on a difference between a current image and a background so as to determine a foreground object;

refining contours of said foreground object image by smoothing to obtain the final image of the animal.

12. The method of claim 9, wherein

identifying all feet of the detected animal comprises using absolute and normalized natural color of the feet to model the feet and using the color models to identify the feet from the rest of the animal.

13. The method of claim 12, wherein identifying all feet of the animal further comprises using statistical shape and relative position information of the feet to assign correspondences for each of the feet in a current image to the instances of the feet from previous images.

14. The method of claim 9, wherein determining the motion of individual feet comprises following successive correspondences from one image to the next and maintaining a list of tracked position and shape data over the course of a sequence of images.

15. The method of claim 9, wherein the images are captured under various types of light conditions including visible, colored and infra-red light.

16. The method of claim 9, wherein the images are captured with the animals in a plurality of cages or arenas, each of which contains a single animal.

17. The method of claim 9, wherein the images are captured with high-speed cameras with speeds of up to 200 frames per second.

18. The method of claim 9, further comprising detecting body parts of the animal.

19. The method of claim 18, further comprising detecting the head of the animal.

20. The method of claim 18, further comprising detecting the tail of the animal.

21. The method of claim 18, further comprising detecting feet of the animal.

26. The method of claim 10, wherein calculating the stride length comprises calculating a distance between an initiation of contact with the tread and a release of contact with the tread between which the foot remains in continuous contact with the tread.

27. The method of claim 10, wherein calculating the stride time comprises calculating the time elapsed between two successive initiations of contact.

28. The method of claim 10, wherein calculating the stance time comprises calculating the time elapsed between initiation and release of contact.

29. The method of claim 10, wherein calculating the swing time comprises calculating the time elapsed between release of contact of one stride and initiation of contact of the successive stride.

30. The method of claim 10, wherein calculating the brake time comprises calculating the time elapsed between an initiation of one stride and an instance when the area of the foot reaches maximum.

31. The method of claim 10, wherein calculating the propulsion time comprises calculating the time elapsed between an instance when the area of the foot reaches maximum during a stance and the release.

32. The method of claim 10, wherein calculating the instantaneous running speed comprises calculating the ratio of the stride length and the stance time, and calculating average running speed comprises calculating the average of the instantaneous running speed over the sequence of images.

33. The method of claim 10, wherein calculating the foot spacing information comprises calculating the distance between midpoints of the strides of following four pairs of feet:

front left foot and front right foot;

rear left foot and rear right foot;

front left foot and rear left foot; and

front right foot and rear right foot.

34. The method of claim 10, wherein calculating the coupling parameters of the feet includes the relative offset expressed as a percentage, of the initiation of the maximum area of the other three feet with respect to the stride of the reference foot.

44. The method of claim 9 wherein the arena is a treadmill apparatus including a transparent movable belt;

and wherein the transparent floor is the transparent movable belt.

35. An animal gait analysis system, comprising

a computer configured to

analyze a plurality of images of at least one animal, wherein the images are images of a bottom view of the animal through a transparent movable belt of a treadmill apparatus;

determine the motion of at least one foot of the animal whether or not the foot is in contact with the transparent belt;

characterize activity of the animal based at least in part on analysis of the motion of the feet.

36. The system of claim 35, further comprising:

a video camera for capturing the images and a video digitization unit coupled to the computer for converting the images from analog to digital format if the images are captured in analog format.

37. The system of claim 35, further comprising:

an animal identification and segregation module receiving the images and identifying and segregating the animal.

38. The system of claim 37, wherein the computer further comprises a foot identification module for segregating the feet from the animal.

39. The system of claim 38, wherein the computer further comprises a foot tracking module for tracking each foot independently.

40. The system of claim 39, wherein the computer further comprises a gait characterization module for extracting gait parameters from the tracked feet.

41. The system of claim 35, wherein the animal is a mouse.

42. The system of claim 35, wherein the animal is a rat.