System and method for judging boundary lines

• US 20020122115A1
• Filed: 12/29/2000
• Published: 09/05/2002
• Est. Priority Date: 12/29/2000
• Status: Active Grant

• Current Assignee: EDH US LLC
• Original Assignee: Auto-Ref Incorporated
• Inventors: Harmath, Miklos
• Granted Patent: US 6,816,185 B2
• Time in Patent Office: Days
• US Class Current: 348/157
• CPC Class Codes: A63B 2102/02 Tennis
• CPC Class Codes: A63B 71/0605 Decision makers and devices using detection means facilitating arbitration

Abstract

There is provided a new and useful system for judging boundary lines on a court in sporting games, for determining whether a ball in play bounces in or out of court. The system comprises at least one camera to capture as frames movement of the ball and a frame grabber associated with each of the cameras to process the frames. An image processing system is configured to receive the digitized frame from the frame grabbers and to process the digitized frame to determine whether the ball is in or out of court and to generate a signal if the ball is out of court. There is an output device coupled to the image processing system to receive the signal and alert the players.

Claims

1. A system for judging boundary lines on a court in sporting games for determining whether a ball in play bounces in or out of court;

said system comprising;

at least one camera to capture movement of said ball;

a frame grabber associated with each of said at least one cameras to process said movement of said ball as captured frames;

an image processing system configured to receive said captured frame from said frame grabbers and to process said captured frame as images to determine whether said ball is in or out of court and to generate a signal if said ball is out of court;

an output device coupled to said image processing system to receive said signal.

2. The system of claim 1 wherein said cameras have a shutter speed of at least {fraction (1/60)}th of a second.

3. The system of claim 1 comprising at most four cameras.

4. The system of claim 1 wherein said cameras are equipped with auto-iris lenses to automatically adjust the aperture of the cameras.

5. The system of claim 1 wherein said cameras may be color or black and white.

6. The system of claim 1 comprising two cameras wherein said cameras are positioned on opposite sides of a centerline transversely dividing said court into two sides, each of said cameras mounted approximately 3 to 5 metres above said court and positioned so as to monitor at least all of the boundary lines on one side of the centerline.

7. The system of claim 6 wherein said cameras are positioned so as to be aligned with said centerline.

8. The system of claim 1 wherein said captured frames comprise the images of said ball recorded by one of said cameras, wherein said images are converted into a continuous line to illustrate the trajectory of the ball.

9. The system of claim 1 wherein said image processing system comprises a calibration sub-system and a recognition sub-system.

10. The system of claim 9 wherein said calibration sub-system comprises a line recognition module, an image segmentation and region grouping module, a court line and region registration module, a supervised ball recognition module and a look-up-table module.

11. The system of claim 10 wherein said line recognition module is configured to recognize and record said boundary lines in said images, using a method of edge detection to correct for distortions.

12. The system of claim 10 wherein said image segmentation and region grouping module is configured to record in and out areas of the court based on input from the user.

13. The system of claim 10 wherein said court line and region registration module is configured to save in and out areas of the court in permanent memory to be used as a comparison point each time the system is commenced.

14. The system of claim 10 wherein said supervised ball recognition module is configured record as a calibrated image a color pattern of the ball placed in the image.

15. The system of claim 10 wherein said look -up-table module is configured to transform input data into output data.

16. The system of claim 15 wherein said look-up-table module is configured as a 256 bytes table.

17. The system of claim 15 wherein said look-up-table module is configured as a mapping function.

18. The system of claim 9 wherein said recognition sub-system comprises a frame/sequence control module, a motion-tracking module, an unsupervised automatic ball recognition module, a trajectory fitting module, a bounce detection module and an in/out judge module.

19. The system of claim 18 wherein said frame/sequence control module is configured to control timing of image capture such that continuous image frames are captured from said frame grabbers.

20. The system of claim 18 wherein said motion-tracking module is configured to locate and isolate moving objects within said image and pass said moving objects to the unsupervised automatic ball recognition module.

21. The system of claim 18 wherein said unsupervised automatic ball recognition module is configured to compare moving objects within said image to a calibrated image of said ball so as to filter out any moving objects which is not the ball, thereby leaving an image of the ball.

22. The system of claim 20 wherein said unsupervised automatic ball recognition module is configured to compare said moving objects within said image to a calibrated image of said ball so as to filter out any moving objects which is not the ball, thereby leaving an image of the ball.

23. The system of claim 18 wherein said trajectory fitting module is configured to fit a path of said ball into a connected trajectory.

24. The system of claim 22 wherein said trajectory fitting module is configured to fit a path of said ball into a connected trajectory.

25. The system of claim 18 wherein said bounce detection module is configured to detect a spot on the court where the ball bounces by locating a sudden change in the direction of the ball.

26. The system of claim 24 wherein said bounce detection module is configured to detect a spot on the court where the ball bounces by locating within said connected trajectory a sudden change in the direction of the ball.

27. The system of claim 18 wherein said in/out judge module is configured to determine whether the ball has bounced in or out of court.

28. The system of claim 26 wherein said in/out judge module is configured to determine whether the ball has bounced in or out of court by comparing the spot on the court where the ball bounces to the boundary lines and determining if the spot overlaps with the lines or does not overlap with the lines.

29. The system of claim 1 further comprising an input device to select a mode of the game and selectively actuate and reset said system.

30. The system of claim 29 wherein said input device comprises voice activated commands.

31. The system of claim 29 wherein said input device comprises manually operable switches.

32. The system of claim 29 wherein said input device comprises remotely operable switches.

33. The system of claim 29 wherein said input device comprises remote touch screen operable switches.

34. The system of claim 29 wherein said input device comprises separate wire connected operable switches.

35. The system of claim 29 wherein said input device and said output device are housed within a single control unit.

36. The system of claim 1 wherein said output device comprises an audible cue, activated upon receipt of said signal.

37. The system of claim 1 wherein said output device is housed within a scoreboard,

38. The system of claim 1 wherein said output device is wired to an existing scoreboard.

39. The system of claim 1 wherein said output device comprises an visual cue, activated upon receipt of said signal.

40. A method of judging boundary lines on a court in sporting games in order to determine whether a ball in play bounces in or out of court, said method utilizing at least one camera to capture movement of said ball, a frame grabber associated with each of said at least one cameras to process said movement as captured frames and an image processing system arranged to receive said captured frame from said frame grabbers and to process said captured frame, said method comprising the steps of:

selecting a mode of game using an input device configured so as to control said system and commence a recognition sub-system;

during play of said game, continuously capturing image frames from said frame grabbers;

within said image frames locating and isolating moving objects;

comparing said moving objects to a calibrated image of said ball so as to filter out any moving object which is not said ball;

fitting a path of said ball into a connected trajectory;

detecting a spot on said court where the ball bounces by locating within said trajectory a sudden change in direction;

determining whether said ball is in or out of court;

transferring a signal to an output device if said ball is determined to be out of court.

41. The method of claim 40 further comprising an initial calibration sequence performed when said system is initially set up at said court or when configuration of said cameras has changed.

42. The method of claim 41 wherein said calibration sequence comprises the steps of recognizing and recording said boundary lines in said images, recording in and out areas of said court, saving said in and out areas in permanent memory, recording a color or grey level range pattern of said ball, and creating a look-up table to transform input data into output data.