Real-time method and apparatus for tracking a moving object experiencing a change in direction

• US 20030142210A1
• Filed: 01/31/2002
• Published: 07/31/2003
• Est. Priority Date: 01/31/2002
• Status: Active Grant

• Current Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Opalach, Agata, Carlbom, Ingrid Birgitta, West, James Edward
• Granted Patent: US 7,030,905 B2
• Time in Patent Office: Days
• US Class Current: 348/157
• CPC Class Codes: A63B 2071/0611 Automatic tennis linesmen, i.e. in-out detectors
• CPC Class Codes: A63B 2220/806 Video cameras
• CPC Class Codes: A63B 71/0605 Decision makers and devices using detection means facilitating arbitration
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 11/14 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 5/22 Position of source determined by co-ordinating a plurality of position lines defined by path-difference measurements G01S5/28 takes precedence
• CPC Class Codes: G01V 1/001 Acoustic presence detection
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

A method and apparatus for tracking an object using one or more video cameras together with a plurality of microphones and/or geophones, whereby accurate tracking of the object can be obtained even when the object changes directions as a result of an impact. A tennis ball may be tracked during a tennis match. Prior to and after an impact, the ball is tracked with the use of video cameras, illustratively in accordance with the method disclosed in U.S. Pat. No. 6,233,007. Then, the position of the ball at racket or ground impact is accurately determined by measuring the acoustical waves generated by the ball hitting the racket and/or either the acoustic waves or the elastic (Rayleigh) waves generated by the ball hitting the ground, respectively. Tracking then advantageously continues with use of the video cameras.

Claims

1. A method for tracking an object using one or more video cameras and a plurality of wave receivers, the video cameras outputting corresponding video images including a sequence of video frames, each of the wave receivers for use in determining an arrival of a wave, the method comprising the steps of:

obtaining a first current frame and a corresponding previous frame from each of said video images, wherein each of said frames includes an image of the object;

determining a first motion vector corresponding to a first direction of motion of the object based on said first current frames and said corresponding previous frames from said video images;

detecting an arrival of a wave by each one of one or more pairs of said wave receivers;

determining one or more time differences between each of said arrivals of said wave at each of said one or more pairs of wave receivers;

determining a location of a change in direction of said object by identifying a source location from which said wave has been generated based on said plurality of time differences;

obtaining a second current frame from each of said video images, wherein each of said frames includes an image of the object; and

determining a second motion vector corresponding to a second direction of motion of the object based on said second current frames from each of said video images and based on said determined location of said change in direction of said object, said second motion vector representing a direction of motion of said object after said change in direction of said object has occurred.

2. The method of claim 1 wherein said wave receivers comprise microphones and wherein said waves comprise acoustic waves.

3. The method of claim 1 wherein said wave receivers comprise geophones and wherein said waves comprise Rayleigh surface waves.

4. The method of claim 1 wherein said change in direction of said object results from a deflection of said object by another object.

5. The method of claim 4 wherein said object comprises a tennis ball, said other object comprises a tennis racket, and said location of a change in direction of said object comprises a location at which said tennis racket impacts said tennis ball.

6. The method of claim 1 wherein said change in direction of said object results from a deflection of said object by a ground surface.

7. The method of claim 6 wherein said object comprises a tennis ball, said ground surface comprises a tennis court playing surface, and said location of a change in direction of said object comprises a location at which said tennis ball impacts said tennis court playing surface.

8. An apparatus for tracking an object using one or more video cameras and a plurality of wave receivers, the video cameras outputting corresponding video images including a sequence of video frames, each of the wave receivers for use in determining an arrival of a wave, the apparatus comprising:

means for obtaining a first current frame and a corresponding previous frame from each of said video images, wherein each of said frames includes an image of the object;

means for determining a first motion vector corresponding to a first direction of motion of the object based on said first current frames and said corresponding previous frames from each of said video images;

means for detecting an arrival of a wave by each one of one or more pairs of said wave receivers;

means for determining one or more time differences between each of said arrivals of said wave at each of one or more pairs of wave receivers;

means for determining a location of a change in direction of said object by identifying a source location from which said wave has been generated based on said plurality of time differences;

means for obtaining a second current frame from each of said plurality of said video images, wherein each of said frames includes an image of the object; and

means for determining a second motion vector corresponding to a second direction of motion of the object based on said second current frames from each of said video images and based on said determined location of said change in direction of said object, said second motion vector representing a direction of motion of said object after said change in direction of said object has occurred.

9. The apparatus of claim 8 wherein said wave receivers comprise microphones and wherein said waves comprise acoustic waves.

10. The apparatus of claim 8 wherein said wave receivers comprise geophones and wherein said waves comprise Rayleigh surface waves.

11. The apparatus of claim 8 wherein said change in direction of said object results from a deflection of said object by another object.

12. The apparatus of claim 11 wherein said object comprises a tennis ball, said other object comprises a tennis racket, and said location of a change in direction of said object comprises a location at which said tennis racket impacts said tennis ball.

13. The apparatus of claim 8 wherein said change in direction of said object results from a deflection of said object by a ground surface.

14. The apparatus of claim 13 wherein said object comprises a tennis ball, said ground surface comprises a tennis court playing surface, and said location of a change in direction of said object comprises a location at which said tennis ball impacts said tennis court playing surface.

15. A method for initiating a process of tracking an object using one or more video cameras and a plurality of wave receivers, the video cameras outputting corresponding video images including a sequence of video frames, each of the wave receivers for use in determining an arrival of a wave, the method comprising the steps of:

detecting an arrival of a wave by each one of one or more pairs of said wave receivers;

determining one or more time differences between each of said arrivals of said wave at each of said one or more pairs of wave receivers;

determining an initial location of said object by identifying a source location from which said wave has been generated based on said plurality of time differences;

obtaining a video frame from each of said video images, wherein each of said frames includes an image of the object; and

determining an initial motion vector corresponding to an initial direction of motion of the object based on said video frames from each of said video images and based on said determined initial location of said object.

16. The method of claim 15 wherein said wave receivers comprise microphones and wherein said,waves comprise acoustic waves.

17. The method of claim 15 wherein said initial location of said object comprises a location at which said object is impacted by another object.

18. The method of claim 17 wherein said object comprises a tennis ball and said other object comprises a tennis racket.

19. An apparatus for initiating a process of tracking an object using one or more video cameras and a plurality of wave receivers, the video cameras outputting corresponding video images including a sequence of video frames, each of the wave receivers for use in determining an arrival of a wave, the apparatus comprising:

means for detecting an arrival of a wave by each one of one or more pairs of said wave receivers;

means for determining one or more time differences between each of said arrivals of said wave at each of said one or more pairs of wave receivers;

means for determining an initial location of said object by identifying a source location from which said wave has been generated based on said plurality of time differences;

means for obtaining a video frame from each of said video images, wherein each of said frames includes an image of the object; and

means for determining an initial motion vector corresponding to an initial direction of motion of the object based on said video frames from each of said video images and based on said determined initial location of said object.

20. The apparatus of claim 19 wherein said wave receivers comprise microphones and wherein said waves comprise acoustic waves.

21. The apparatus of claim 19 wherein said initial location of said object comprises a location at which said object is impacted by another object.

22. The apparatus of claim 21 wherein said object comprises a tennis ball and said other object comprises a tennis racket.

23. A method for terminating a process of tracking an object using one or more video cameras and a plurality of wave receivers, the video cameras outputting corresponding video images including a sequence of video frames, each of the wave receivers for use in determining an arrival of a wave, the method comprising the steps of:

obtaining a current frame and a corresponding previous frame from each of said video images, wherein each of said frames includes an image of the object;

determining a motion vector corresponding to a direction of motion of the object based on said current frames and said corresponding previous frames from said video images;

detecting an arrival of a wave by each one of one or more pairs of said wave receivers;

determining one or more time differences between each of said arrivals of said wave at each of said one or more pairs of wave receivers;

determining a location of said object by identifying a source location from which said wave has been generated based on said plurality of time differences; and

terminating said tracking process based on the determined location of said object.

24. The method of claim 23 wherein said wave receivers comprise microphones and wherein said waves comprise acoustic waves.

25. The method of claim 23 wherein said wave receivers comprise geophones and wherein said waves comprise Rayleigh surface waves.

26. The method of claim 23 wherein said determined location of said object is located on a ground surface.

27. The method of claim 26 wherein said object comprises a tennis ball, said ground surface comprises a tennis court playing surface, and said determined location of said object comprises a location at which said tennis ball impacts said tennis court playing surface.

28. An apparatus for terminating a process of tracking an object using one or more video cameras and a plurality of wave receivers, the video cameras outputting corresponding video images including a sequence of video frames, each of the wave receivers for use in determining an arrival of a wave, the apparatus comprising:

means for obtaining a current frame and a corresponding previous frame from each of said video images, wherein each of said frames includes an image of the object;

means for determining a motion vector corresponding to a direction of motion of the object based on said current frames and said corresponding previous frames from said video images;

means for detecting an arrival of a wave by each one of one or more pairs of said wave receivers;

means for determining one or more time differences between each of said arrivals of said wave at each of said one or more pairs of wave receivers;

means for determining a location of said object by identifying a source location from which said wave has been generated based on said plurality of time differences; and

means for terminating said tracking process based on the determined location of said object.

29. The apparatus of claim 28 wherein said wave receivers comprise microphones and wherein said waves comprise acoustic waves.

30. The apparatus of claim 28 wherein said wave receivers comprise geophones and wherein said waves comprise Rayleigh surface waves.

31. The apparatus of claim 28 wherein said determined location of said object is located on a ground surface.

32. The apparatus of claim 31 wherein said object comprises a tennis ball, said ground surface comprises a tennis court playing surface, and said determined location of said object comprises a location at which said tennis ball impacts said tennis court playing surface.