Motion information extraction system

• US 6,757,328 B1
• Filed: 12/15/1999
• Issued: 06/29/2004
• Est. Priority Date: 05/28/1999
• Status: Expired due to Fees

• Current Assignee: Kent Ridge Digital Labs
• Original Assignee: Kent Ridge Digital Labs
• Inventors: Huang, Qingming, Lo, Sheng, Tian, Qi
• Primary Examiner(s): Kelley, Chris
• Assistant Examiner(s): SENFI, BEHROOZ M
• Application Number:
  US09/464,533
• Time in Patent Office: 1,658 Days
• Field of Search: 348/415.1, 348/416.1, 348/401.1, 348/420.1, 348/459, 348/443, 348/444, 348/456, 382/307, 382/104, 382/141, 382/305, 375/240.1, 375/240.02
• US Class Current: 375/240.1
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: H04N 19/513 Processing of motion vectors

Abstract

A method, an apparatus, and a computer program product for extracting motion information (110) from a video sequence (130, 600) containing interframe motion vectors (120) are disclosed. In particular, motion information (110) is automatically extracted (610) from an encoded traffic video stream (600) to detect speed, density and flow. The motion information (110) extracted is under fixed camera settings and in a well-defined environment. The motion vectors (120) are first separated (610) from the compressed streams (130) during decoding and filtered (620) to eliminate incorrect and noisy motion vectors based on the well-defined environmental knowledge. By applying a projective transform (630) to the filtered motion vectors, speed, density, and flow can be detected (640, 650, 660). In this manner, a traffic monitoring system is implemented.

Claims

1. A method of extracting motion information of an object in a video sequence from an encoded video stream containing interframe motion vectors under fixed camera settings and a well defined environment, said method including the steps of:

separating motion vectors obtained from said encoded video stream;

filtering said motion vectors based on predetermined environmental knowledge; and

determining predetermined parameters of the object based on said filtered motion vectors, wherein said predetermined parameters of the object are mapped to world coordinates from image coordinates by a projective transform.

2. The method according to claim 1, wherein said filtering step includes the sub-step of eliminating any motion vectors that:

do not coincide with a road direction, intersect with other motion vectors, cross a road border, or do not have appropriate amplitude or size.

3. The method according to claim 1, wherein said predetermined parameters include speed, density and flow.

4. The method according to claim 1, wherein said predetermined parameters are provided at regular time intervals.

5. The method according to claim 1, wherein said encoded video stream is obtained from a sky camera.

6. The method according to claim 1, wherein said determining step includes the step of calculating said motion information using motion vector analysis on said filtered motion vectors.

7. The method according to claim 1, wherein said video stream is a motion vector presentation of compressed video.

8. The method according to claim 7, wherein said compressed video has a format selected from the group of formats consisting of MPEG and H.26x.

9. The method according to claim 1, further including the step of detecting speed based on an amplitude calculation of said filtered motion vectors.

10. The method according to claim 1, further including the step of detecting density based on occupancy computation of microblocks with nonzero motion vectors.

11. The method according to claim 1, further including the step of estimating flow based on a combination of speed and density detection, said speed detected based on an amplitude calculation of said filtered motion vectors and said density detected by an occupancy computation of microblocks with nonzero motion vectors.

12. The method according to claim 1, wherein said interframe motion vectors are generated using electronic encoding hardware.

13. The method according to claim 1, wherein said filtering step includes at least one of the sub-steps of:

eliminating any motion vectors that do not coincide with a predetermined direction;

eliminating any motion vectors that intersect with other motion vectors;

eliminating any motion vectors that cross a predetermined border; and

eliminating any motion vectors that do not have an appropriate amplitude or size.

14. An apparatus for extracting motion information of an object in a video sequence from an encoded video stream containing interframe motion vectors under fixed camera settings and a well defined environment, said apparatus including:

means for separating motion vectors obtained from said encoded video stream;

means for filtering said motion based on predetermined environmental knowledge; and

means for determining predetermined parameters of the object based on said filtered motion vectors, wherein said predetermined parameters of the object are mapped to world coordinates from image coordinates by a projective transform.

15. The apparatus according to claim 14, wherein said filtering means includes means for eliminating any motion vectors that:

do not coincide with a road direction, intersect with other motion vectors, cross a road border, or do not have appropriate amplitude or size.

16. The apparatus according to claim 14, wherein said predetermined parameters include speed, density and flow.

17. The apparatus according to claim 14, wherein said predetermined parameters are provided at regular time intervals.

18. The apparatus according to claim 14, wherein said encoded video stream is obtained from a sky camera.

19. The apparatus according to claim 14, wherein said determining means includes means for calculating said motion information using motion vector analysis on said filtered motion vectors.

20. The apparatus according to claim 14, wherein said video stream is a motion vector presentation of compressed video.

21. The apparatus according to claim 20, wherein said compressed video has a format selected from the group of formats consisting of MPEG and H.26x.

22. The apparatus according to claim 14, further including means for detecting speed based on an amplitude calculation of said filtered motion vectors.

23. The apparatus according to claim 14, further including means for detecting density based on occupancy computation of microblocks with nonzero motion vectors.

24. The apparatus according to claim 14, further including means for estimating flow based on a combination of speed and density detection, said speed detected based on an amplitude calculation of said filtered motion vectors and said density detected by an occupancy computation of microblocks with nonzero motion vectors.

25. The apparatus according to claim 14, wherein said interframe motion vectors are generated using electronic encoding hardware.

26. The apparatus according to claim 14, wherein said filtering means includes at least one of:

means for eliminating any motion vectors that do not coincide with a predetermined direction;

means for eliminating any motion vectors that intersect with other motion vectors;

means for eliminating any motion vectors that cross a predetermined border; and

means for eliminating any motion vectors that do not have an appropriate amplitude or size.

27. A computer program product having a computer readable medium having a computer program recorded therein for extracting motion information of an object in a video sequence from an encoded video stream containing interframe motion vectors under fixed camera settings and a well defined environment, said computer program product including:

means for separating motion vectors obtained from said encoded video stream;

means for filtering said motion vectors based on predetermined environmental knowledge; and

means for determining predetermined parameters of the object based on said filtered motion vectors, wherein said predetermined parameters of the object are mapped to world coordinates from image coordinates by a projective transform.

28. The computer program product according to claim 27, wherein said filtering means includes means for eliminating any motion vectors that:

do not coincide with a road direction, intersect with other motion vectors, cross a road border, or do not have appropriate amplitude or size.

29. The computer program product according to claim 27, wherein said predetermined parameters include speed, density and flow.

30. The computer program product according to claim 27, wherein said predetermnined parameters are provided at regular time intervals.

31. The computer program product according to claim 27, wherein said encoded video stream is obtained from a sky camera.

32. The computer program product according to claim 27, wherein said determining means includes means for calculating said motion information using motion vector analysis on said filtered motion vectors.

33. The computer program product according to claim 27, wherein said video stream is a motion vector presentation of compressed video.

34. The computer program product according to claim 33, wherein said compressed video has a format selected from the group of formats consisting of MPEG and H.26x.

35. The computer program product according to claim 27, further including means for detecting speed based on an amplitude calculation of said filtered motion vectors.

36. The computer program product according to claim 27, further including means for detecting density based on occupancy computation of microblocks with nonzero motion vectors.

37. The computer program product according to claim 27, further including means for estimating flow based on a combination of speed and density detection, said speed detected based on an amplitude calculation of said filtered motion vectors and said density detected by an occupancy computation of microblocks with nonzero motion vectors.

38. The computer program product according to claim 27, wherein said interframe motion vectors are generated using electronic encoding hardware.

39. The computer program product according to claim 27, wherein said filtering means includes at least one of:

means for eliminating any motion vectors that do not coincide with a predetermined direction;

means for eliminating any motion vectors that intersect with other motion vectors;

means for eliminating any motion vectors that cross a predetermined border; and

means for eliminating any motion vectors that do not have an appropriate amplitude or size.