System and method for judging boundary lines

• US 6,816,185 B2
• Filed: 12/29/2000
• Issued: 11/09/2004
• Est. Priority Date: 12/29/2000
• Status: Expired due to Term

• Current Assignee: EDH US LLC
• Original Assignee: Auto-Ref Incorporated
• Inventors: Harmath, Miki
• Primary Examiner(s): PHILIPPE, GIMS S
• Application Number:
  US09/750,995
• Publication Number:
  US 20020122115A1
• Time in Patent Office: 1,411 Days
• Field of Search: 348/157, 348/169, 348/154-155, 348/159, 340/323.R, 340/33.D, 340/815.4, 340/384.1
• US Class Current: 348/157
• CPC Class Codes: A63B 2102/02 Tennis
• CPC Class Codes: A63B 71/0605 Decision makers and devices using detection means facilitating arbitration

Abstract

There is provided a new and useful system for judging boundary lines on a court in sporting games, for determining whether a ball in play bounces in or out of court. The system comprises at least one camera to capture as frames movement of the ball and a frame grabber associated with each of the cameras to process the frames. An image processing system is configured to receive the digitized frame from the frame grabbers and to process the digitized frame to determine whether the ball is in or out of court and to generate a signal if the ball is out of court. There is an output device coupled to the image processing system to receive the signal and alert the players.

Claims

1. A system for judging boundary lines on a court in sporting games for determining whether a ball in play bounces in or out of court;

said system comprising;

at least one camera positioned proximate to the court independent of the orientation of said boundary lines to capture movement of said ball;

a frame grabber associated with each of said at least one cameras to process said movement of said ball as captured frames;

an image processing system configured to receive said captured frame from said frame grabbers and to process said captured frame as images to determine whether said ball is in or out of court and to generate a signal if said ball is out of court, said image processing system comprising a calibration sub-system and a recognition sub-system, wherein said calibration sub-system comprises a line recognition module, an image segmentation and region grouping module, a court line and region registration module, a supervised ball recognition module and a look-up-table module; and

an output device coupled to said image processing system to receive said signal.

2. The system of claim 1 wherein said line recognition module is configured to recognize and record said boundary lines in said images, using a method of edge detection to correct for distortions.

3. The system of claim 1 wherein said image segmentation and region grouping module is configured to record in and out areas of the court based on input from the user.

4. The system of claim 1 wherein said court line and region registration module is configured to save in and out areas of the court in permanent memory to be used as a comparison point each time the system is commenced.

5. The system of claim 1 wherein said supervised ball recognition module is configured record as a calibrated image a color pattern of the ball placed in the image.

6. The system of claim 1 wherein said look-up-table module is configured to transform input data into output data.

7. The system of claim 6 wherein said look-up-table module is configured as a 256 bytes table.

8. The system of claim 6 wherein said look-up-table module is configured as a mapping function.

9. A system for judging boundary lines on a court in sporting games for determining whether a ball in play bounces in or out of court;

said system comprising;

at least one camera positioned proximate to the court independent of the orientation of said boundary lines to capture movement of said ball;

a frame grabber associated with each of said at least one cameras to process said movement of said ball as captured frames;

an image processing system configured to receive said captured frame from said frame grabbers and to process said captured frame as images to determine whether said ball is in or out of court and to generate a signal if said ball is out of court, said image processing system comprising a calibration sub-system and a recognition sub-system, wherein said recognition sub-system comprises a frame/sequence control module, a motion-tracking module, an unsupervised automatic ball recognition module, a trajectory fitting module, a bounce detection module and an in/out judge module; and

an output device coupled to said image processing system to receive said signal.

10. The system of claim 9 wherein said frame/sequence control module is configured to control timing of image capture such that continuous image frames are captured from said frame grabbers.

11. The system of claim 9 wherein said motion-tracking module is configured to locate and isolate moving objects within said image and pass said moving objects to the unsupervised automatic ball recognition module.

12. The system of claim 9 wherein said unsupervised automatic ball recognition module is configured to compare moving objects within said image to a calibrated image of said ball so as to filter out any moving objects which is not the ball, thereby leaving an image of the ball.

13. The system of claim 11 wherein said unsupervised automatic ball recognition module is configured to compare said moving objects within said image to a calibrated image of said ball so as to filter out any moving objects which is not the ball, thereby leaving an image of the ball.

14. The system of claim 9 wherein said trajectory fitting module is configured to fit a path of said ball into a connected trajectory.

15. The system of claim 13 wherein said trajectory fitting module is configured to fit a path of said ball into a connected trajectory.

16. The system of claim 9 wherein said bounce detection module is configured to detect a spot on the court where the ball bounces by locating a sudden change in the direction of the ball.

17. The system of claim 15 wherein said bounce detection module is configured to detect a spot on the court where the ball bounces by locating within said connected trajectory a sudden change in the direction of the ball.

18. The system of claim 9 wherein said in/out judge module is configured to determine whether the ball has bounced in or out of court.

19. The system of claim 17 wherein said in/out judge module is configured to determine whether the ball has bounced in or out of court by comparing the spot on the court where the ball bounces to the boundary lines and determining if the spot overlaps with the lines or does not overlap with the lines.

20. A system for judging boundary lines on a court in sporting games for determining whether a ball in play bounces in or out of court;

said system comprising;

at least one camera positioned proximate to the court independent of the orientation of said boundary lines to capture movement of said ball;

a frame grabber associated with each of said at least one cameras to process said movement of said ball as captured frames;

an image processing system configured to receive said captured frame from said frame grabbers and to process said captured frame as images to determine whether said ball is in or out of court and to generate a signal if said ball is out of court;

an output device coupled to said image processing system to receive said signal; and

an input device to select a mode of the game and selectively actuate and reset said system.

21. The system of claim 20 wherein said input device comprises voice activated commands.

22. The system of claim 20 wherein said input device comprises manually operable switches.

23. The system of claim 20 wherein said input device comprises remotely operable switches.

24. The system of claim 20 wherein said input device comprises remote touch screen operable switches.

25. The system of claim 20 wherein said input device comprises separate wire connected operable switches.

26. The system of claim 20 wherein said input device and said output device are housed within a single control unit.

27. The system of claim 20 wherein said output device comprises an audible cue, activated upon receipt of said signal.

28. The system of claim 20 wherein said output device is housed within a scoreboard.

29. The system of claim 20 wherein said output device is wired to an existing scoreboard.

30. The system of claim 20 wherein said output device comprises an visual cue, activated upon receipt of said signal.

31. A method of judging boundary lines on a court in sporting games in order to determine whether a ball in play bounces in or out of court, said method utilizing at least one camera to capture movement of said ball, a frame grabber associated with each of said at least one cameras to process said movement as captured frames and an image processing system arranged to receive said captured frame from said frame grabbers and to process said captured frame, said method comprising the steps of:

selecting a mode of game using an input device configured so as to control said system and commence a recognition sub-system;

during play of said game, continuously capturing image frames from said frame grabbers;

within said image frames locating and isolating moving objects;

comparing said moving objects to a calibrated image of said ball so as to filter out any moving object which is not said ball;

fitting a path of said ball into a connected trajectory;

detecting a spot on said court where the ball bounces by locating within said trajectory a sudden change in direction;

determining whether said ball is in or out of court;

transferring a signal to an output device if said ball is determined to be out of court.

32. The method of claim 31 further comprising an initial calibration sequence performed when said system is initially set up at said court or when configuration of said cameras has changed.

33. The method of claim 32 wherein said calibration sequence comprises the steps of recognizing and recording said boundary lines in said images, recording in and out areas of said court, saving said in and out areas in permanent memory, recording a color or grey level range pattern of said ball, and creating a look-up table to transform input data into output data.