Calibration system for vision-based automatic writing implement

• US 6,885,759 B2
• Filed: 03/30/2001
• Issued: 04/26/2005
• Est. Priority Date: 03/30/2001
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Cox, Cory W, Lantz, Philip R
• Primary Examiner(s): Patel, Kanjibhai
• Assistant Examiner(s): Bayat, Ali
• Application Number:
  US09/822,922
• Publication Number:
  US 20020141616A1
• Time in Patent Office: 1,488 Days
• Field of Search: 382/103, 382/106, 382151-153, 382/186, 382313-315, 382187-189, 348/169, 348/113, 348/116, 348/170, 348/171, 348/175, 356/3, 235/411
• US Class Current: 382/103
• CPC Class Codes: G06T 1/0014 Image feed-back for automatic industrial control, e.g. robot with camera robots B25J19/023
• CPC Class Codes: G06T 2207/30144 Printing quality
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning

Abstract

A calibration system for a vision-based automatic writing implement is disclosed. The automatically controlled writing implement is appended to an articulated robot arm. The writing implement draws or writes on a substrate and recognizes what a user draws or writes on the same substrate. The user may move the substrate while drawing, thereby confusing the drawing and recognition processes. Accordingly, a controller draws a target on the substrate initially. Before each drawing or recognition step, the controller uses a digital video camera to find and recognize the target and determine whether the substrate has moved. The target is asymmetrical, such that the controller can determine the orientation as well as the position of the substrate. The controller can then orient its drawing and recognition algorithms to the new location and orientation of the substrate.

Claims

1. A method to detect substrate movement, the method comprising:

detecting with an image sensor that a substrate is blank;

drawing a specialized target on the substrate; and

using the image sensor to find and recognize the target.

2. The method of claim 1, wherein the image sensor is placed in a position where it can observe the substrate while a user is marking the substrate.

3. The method of claim 2, wherein the image sensor searches for the target in a plurality of video frames.

4. The method of claim 3, wherein a determination as to whether the substrate moved is made.

5. The method of claim 4, wherein the relative positions of the target and a drawing on the substrate are remembered, and this information is used to locate the drawing in the video frame is located relative to the target.

6. The method of claim 5, wherein the drawing in the video frame is examined for determining whether the user marked the substrate.

7. The method of claim 6, wherein a subsequent drawing is made by using the position of the target to find the location of the substrate suitable for drawing.

8. The method of claim 7, wherein the process beginning with allowing the user to mark the substrate is repeated.

9. The method of claim 6, wherein the steps of searching for the target in a plurality of video frames, determining whether the substrate moved, locating the drawing in the video frame relative to the target, and examining the drawing in the video frame are repeated until a determination that the user marked the substrate is made.

10. The method of claim 1, wherein the image sensor is a digital video camera.

11. The method of claim 1, wherein a controller draws the specialized target on the substrate.

12. The method of claim 1, wherein the target is asymmetrical to enable the controller to determine the orientation and position of the substrate.

13. A computer readable medium having computer readable instructions encoded thereon for:

drawing a specialized target on a blank substrate;

determining the orientation and the position of the substrate;

receiving video data from a digital video camera;

performing a frame-by-frame analysis on the data to detect the presence of the specialized target, the frame-by-frame analysis being performed in a plurality of video frames;

performing the frame-by-frame analysis on the plurality of video frames to detect the presence of drawings or writings in a video frame relative to the specialized target; and

examining the drawing or writing in the plurality of video frames to determine whether a user marked the substrate.

14. The computer readable medium of claim 13 wherein a subsequent drawing is made by using the position of the target to find the location of the substrate suitable for drawing.

15. The computer readable medium of claim 14, wherein the process beginning with allowing the user to mark the substrate is repeated.

16. The computer readable medium of claim 13, wherein the steps of searching for the target in a plurality of video frames, locating the drawing in the video frame relative to the target, and examining the drawing in the video frame are repeated until a determination that the user marked the substrate is made.

17. The computer readable medium of claim 13, the computer readable medium further including computer readable instructions encoded thereon for remembering the relative positions of the target and drawings on the substrate.

18. An apparatus comprising:

an articulated robot arm for drawing a specialized target on a substrate and for drawing or writing secured to a base;

an image sensor for finding and recognizing the target; and

a controller, having a storage medium, including a rule set to interact with a user and instructions to manipulate the robot arm,

to receive and process an image from the image sensor;

to determine an action based on the rule set and the image; and

to move the robot arm based on a determination of the image based on the rule set.

19. The apparatus of claim 18, wherein the specialized target is asymmetrical.

20. The apparatus of claim 18, wherein the relative positions of the target and a drawing on the substrate are remembered, and this information is used to locate the drawing in the video frame relative to the target.

21. The apparatus of claim 18, wherein the robot arm comprises a plurality of software-controlled mechanical actuators and sensors.

22. The apparatus of claim 18, wherein the image sensor is a digital video camera that provides video to the controller.