Method and apparatus for positional error correction in a robotic pool systems using a cue-aligned local camera

• US 20060063599A1
• Filed: 09/23/2005
• Published: 03/23/2006
• Est. Priority Date: 09/23/2004
• Status: Active Grant

• Current Assignee: Michael Greenspan
• Original Assignee: Michael Greenspan
• Inventors: Greenspan, Michael
• Granted Patent: US 7,831,337 B2
• Time in Patent Office: Days
• US Class Current: 473/1
• CPC Class Codes: A63D 15/00 Billiards, e.g. carom billiards or pocket billiards; Billiard tables bagatelle A63D13/00
• CPC Class Codes: B25J 9/1697 Vision controlled systems

Abstract

The present invention provides a method and apparatus for accurately positioning a robotic pool-playing device. The system comprises a computer controlled robotic positioning device, such as a gantry robot, that can position a cue over the pool table and place a shot. A global camera is mounted on the ceiling looking down at the table, and the acquired images are transmitted to the computer for analysis to determine the identity and locations of the balls within the table coordinate reference frame. The computer also automatically determines which ball to strike. An aspect of the invention is the use of a local camera, mounted on or near the robotic end-effector in a fixed relationship with the cue, to improve the positioning error of the robotic device prior to placing a shot. By comparing the ball locations perceived from the vantage of the local camera with the known ball locations determined from the global camera image, the invention can calculate the acquired robotic positioning error, which can then be corrected for prior to placing the shot.

Claims

1. A method of playing pool, comprising the steps of:

a) acquiring an image of a pool table using a first camera placed above the pool table and positioned that its image plane is substantially parallel to both a playing surface of the pool table and a longitudinal axis of a pool cue and transmitting this image to a computer processing means for analysis, the result of which is a determination of the position and identity of each pool ball on the pool table;

b) said computer processing means planning a shot of a cue ball by the pool cue by calculating a desired position of the pool cue with respect to the cue ball in a pool table frame of reference including a tip position of said pool cue and orientation of the longitudinal axis of the pool cue;

c) said computer processing means instructing said robot connected to said pool cue to position the pool cue to a preferred location from which to place a shot; and

d) said computer processing means instructing the robot to place a shot.

2. The method according to claim 1 wherein after step c) and prior to step d), including a step of acquiring a second image of a pool table using a second camera mounted on the robotic end-effector so that its image plane is substantially perpendicular to both the pool table and said longitudinal axis of the pool cue, the position of said second camera being fixed with respect to said longitudinal axis of the pool cue, transmitting the second image to the computer processing means and determining locations of some of the pool balls within this image within said second camera frame of reference, and wherein if at least three pool balls are visible from both the first and second cameras, then estimating and correcting for an accumulated positional error of the robotic system and repositioning the pool cue by moving at least one proximal robotic joint which also repositions the second camera.

3. The method according to claim 1 wherein after step c) and prior to step d), including a step of acquiring a second image of a pool table using a second camera mounted on the robotic end-effector so that its image plane is substantially perpendicular to both the pool table and said longitudinal axis of the pool cue, the position of said second camera being fixed with respect to said longitudinal axis of the pool cue, transmitting the second image to the computer processing means and determining locations of some of the pool balls within this image within a second camera frame of reference, and wherein if less than three pool balls are visible from either or both of the first and second cameras, then estimating and correcting for some but not necessarily all components of the accumulated positional error of the robotic system, and repositioning the pool cue by moving at least one proximal robotic joint which also repositions the second camera.

4. The method according to claim 2 wherein the location of the longitudinal axis of the cue is determined within a coordinate reference frame of the second camera.

5. The method according to claim 3 wherein the location of the longitudinal axis of the cue is determined within a coordinate reference frame of the second camera.

6. The method according to claim 4 wherein the position of the second camera is fixed with respect to the longitudinal cue axis, and the location of the cue axis within the coordinate reference frame of the second camera is determined in a preprocessing step including capturing an image using a third camera mounted so that its image plane is substantially perpendicular to the longitudinal axis of the cue, and including re-positioning the pool cue with one degree-of-freedom by sliding along the cue axis repeatedly and the center of the cue tip within the third camera image is determined at each extreme cue position, and wherein by comparing the centers of the tip at each extreme position, computing the angle between the cue axis and the third camera whereupon the robotic joints are repositioned so as to improve the perpendicularity of the cue axis with the third camera image plane, and the process iterates, and once the cue is thus positioned so that its axis is accurately perpendicular to the third camera image plane, the relative locations of the third camera and the local camera are then determined by estimating a rigid transformation that relates them, and the cue location is then directly propagated to the local camera coordinate reference frame.

7. The method according to claim 5 wherein the position of the second camera is fixed with respect to the longitudinal cue axis, and the location of the longitudinal axis of the cue within the coordinate reference frame of the second camera is determined in a preprocessing step including capturing an image using a third camera mounted so that its image plane is substantially perpendicular to the longitudinal axis of the cue, and including re-positioning the pool cue with one degree-of-freedom by sliding along the cue axis repeatedly and the center of the cue tip within the third camera image is determined at each extreme cue position, and wherein by comparing the centers of the tip at each extreme position, computing the angle between the cue axis and the third camera whereupon the robotic joints are repositioned so as to improve the perpendicularity of the cue axis with the third camera image plane, and the process iterates, and once the cue is thus positioned so that its axis is accurately perpendicular to the third camera image plane, the relative locations of the third camera and the local camera are then determined by estimating a rigid transformation that relates them, and the cue location is then directly propagated to the local camera coordinate reference frame.

8. The method according to claim 6 wherein the position of the second camera is fixed with respect to the longitudinal cue axis, and the location of the cue ball and object ball within the coordinate reference frame of the local camera is determined in a preprocessing step, and an estimation and correction of some components of an accumulated positional error of the robot is substantially achieved by matching the current positions of the cue and object balls with those positions that were respectively determined in the preprocessing step.

9. The method according to claim 7 wherein the position of the second camera is fixed with respect to the longitudinal cue axis, and the location of the cue ball and object ball within the coordinate reference frame of the local camera is determined in a preprocessing step, and an estimation and correction of some components of an accumulated positional error of the robot is substantially achieved by matching the current positions of the cue and object balls with those positions that were respectively determined in the preprocessing step.

10. The method according to claim 1 wherein after step c) and prior to step d), including a step of acquiring a second image of a pool table using a second camera mounted on the robotic end-effector so that its image plane is substantially perpendicular to both the pool table and said longitudinal axis of the pool cue, the position of said second camera being fixed with respect to said longitudinal axis of the pool cue, transmitting the second image to the computer processing means and determining locations of some landmarks within this image within said second camera frame of reference, and then estimating and correcting for an accumulated positional error of the robotic system and repositioning the pool cue by moving at least one proximal robotic joint which also repositions the second camera.

11. The method according to claim 10 wherein locations of the landmarks have been predetermined with respect to the first camera frame of reference.

12. A method of playing pool, comprising the steps of;

a) acquiring an image of a pool table using a camera mounted on the robotic end-effector so that its image plane is substantially perpendicular to a longitudinal axis of the pool cue, the position of said camera being fixed with respect to the longitudinal axis of the pool cue, transmitting the image to a computer processing means for analysis, the result of which is a determination of the position and identity of each ball on the pool table within this image within a frame of reference of the camera, and correcting for any errors based upon the relative position of the cue axis and the cue ball;

b) said computer processing means planning a shot of a cue ball by the pool cue by calculating a desired position of the cue with respect to the cue ball; and

c) said computer processing means instructing the robot connected to said pool cue to place a shot.

13. An apparatus for playing pool, comprising;

a) a robot having a robotic arm with an end effector, the robot mounted on a gantry for movement above a pool table, a pool cue attached to the end-effector, the pool cue having a longitudinal axis;

b) image capturing means mounted above the pool table for acquiring an image of the pool table and positioned such that its image plane is substantially parallel to both a playing surface of the pool table and said longitudinal axis of said pool cue;

c) a computer processing means connected to said image capturing means for analysis of the images transmitted thereto from the image capturing means, the computer processing means including means for determining a position and identity of each pool ball on the pool table within this image within a frame of reference of the image capturing means, and including means for correcting for any errors based upon the relative position of the cue axis and the cue ball, said computer processing means including means for planning a shot of a cue ball by the pool cue by calculating a desired position of the cue with respect to the cue ball; and

d) a robotic controller connected to said computer processing means for instructing the robot to place pool shots with the cue.

14. The apparatus for playing pool according to claim 13 wherein said image capturing means is a first image capturing means, including a second image capturing means mounted on the robotic end-effector so that its image plane is substantially perpendicular to the longitudinal axis of the pool cue for capturing an image of a pool table using a camera, the position of said camera being fixed with respect to the longitudinal axis of the pool cue.

15. The apparatus for playing pool according to claim 14 including a third camera mounted such that its image plane is also substantially perpendicular to the longitudinal axis of said cue, the computer processing means includes means for determining a location of the longitudinal axis of said cue within a coordinate reference frame of said second image capturing means.

16. The apparatus for playing pool according to claim 14 including landmarks positioned at specific locations with respect to the pool table wherein the positions of the landmarks can therefore be identified within the frame of reference of the image capturing means, whereby once the cue has been positioned, then the landmark locations can be identified within the frame of reference of the second image capturing means, and a positional error of the cue can then be estimated.

17. The apparatus according to claim 16 wherein locations of the landmarks have been predetermined with respect to the frame of reference of the first image capturing means.

18. An apparatus for playing pool, comprising;

a) a robot having a robotic arm with an end effector, the robot mounted on a gantry for movement above a pool table, a pool cue attached to the end-effector, the cue having a longitudinal axis;

b) image capturing means mounted on the robotic end-effector so that its image plane is substantially perpendicular to a longitudinal axis of the pool cue for capturing an image of a pool table using a camera, the position of said camera being fixed with respect to the longitudinal axis of the pool cue,

c) a computer processing means for analysis of the images transmitted thereto from the image capturing means, the result of which is a determination of the position and identity of each ball on the pool table within this image within a frame of reference of the camera, and correcting for any errors based upon the relative position of the cue axis and the cue ball, said computer processing means planning a shot of a cue ball by the pool cue by calculating a desired position of the cue with respect to the cue ball; and

d) a robotic controller connected to said computer processing means for instructing the robot to place pool shots with the cue.

19. The apparatus for playing pool according to claim 18 including wherein the image capturing means is a first camera, including a second camera mounted above the pool table and positioned so that its image plane is substantially parallel to both the pool table and the longitudinal axis of the pool cue and means for transmitting images from the second camera to the computer processing means.

20. The apparatus for playing pool according to claim 19 including a third camera mounted such that its image plane is also substantially perpendicular to the longitudinal axis of said cue, the computer processing means includes means for determining a location of the longitudinal axis of said cue within a coordinate reference frame of said second image capturing means.