Dynamically Controlling a Cursor on a Screen when Using a Video Camera as a Pointing Device

• US 20090046061A1
• Filed: 08/14/2007
• Published: 02/19/2009
• Est. Priority Date: 08/14/2007
• Status: Active Grant

• Current Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Inventors: Denoue, Laurent, Golovchinsky, Gene, Adcock, John E.
• Granted Patent: US 8,144,123 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

A system provides for controlling a cursor on a screen automatically and dynamically when using a video camera as a pointing device. A computer displays static or dynamic content to a screen. A video camera connected to the computer points at the screen. As the video camera films the screen, frames captured by the video camera are sent to the computer. A target image is displayed by the computer onto the screen and marks the position of the screen cursor of the video camera. Frames captured by the video camera include the target image, and the computer dynamically moves the target image on the screen to ensure that the target image stays in the center of the view of the video camera.

Claims

1. A computer-based method for automatically and dynamically controlling a cursor on a screen when using a video camera as a pointing device, the method comprising:

displaying by the computer dynamic content to the screen connected to the computer;

filming of the screen by the video camera, the video camera pointing at the screen and connected to the computer;

displaying by the computer a target image to the screen such that frames captured by the video camera include the target image; and

dynamically moving the target image on the screen by the computer to ensure that the target image stays in the center of the view of the video camera, the target image marking the position of the screen cursor of the video camera pointing device.

2. The computer-based method of claim 1, further comprising changing by the computer one of the following to ensure the target image remains detectable:

the display around the target image; and

the target image itself.

3. The computer-based method of claim 1, further comprising dynamically adjusting by the computer the size and opacity of the target image so that the target image remains detectable when the video camera is moved away from the screen.

4. The computer-based method of claim 1, further comprising estimating by the computer a distance between the video camera and the screen, the estimation used in a computation for moving the target image.

5. The computer-based method of claim 1, further comprising estimating by the computer a position of the video camera relative to the screen based on one or more of yaw, pitch, and roll of the video camera, the estimation used in a computation for moving the target image.

6. The computer-based method of claim 1, further comprising manually initializing the system by a user that picks the target image by pointing the video camera at the target image.

7. The computer-based method of claim 1, further comprising automatically initializing the system by displaying the target image onto different areas of the screen until the target image appears in a frame captured by the video camera.

8. The computer-based method of claim 1, further comprising automatically initializing the system by predicting where to move the target image by a recursive function that determines a first quadrant of the screen that appears in a frame captured by the video camera, then determines a second quadrant of the first quadrant that appears in the frame, and so on, until the quadrant size is equal to the target image size.

9. The computer-based method of claim 1, further comprising automatically initializing the system by displaying the target image to a predefined area on the screen such that the target image appears in the frame captured by the video camera pointed at the predefined area.

10. The computer-based method of claim 1, further comprising identifying by the computer for one or more additional screens where the screens are located in relation to each other.

11. The computer-based method of claim 10, wherein identifying where the screens are located in relation to each other further comprises determining if the video camera that points to a first screen is moved to point to a second adjacent screen, that the target image follows from the first screen to the second screen.

12. The computer-based method of claim 1, further comprising controlling by one or more additional video cameras of their respective cursors, each cursor associated with a target image, and each target image distinguished by one or more of color, shape, and texture.

14. The system of claim 1, further comprising a change by the computer of one of the following to ensure the target image remains detectable:

the display around the target image;

and the target image itself.

15. The system of claim 1, further comprising a dynamic adjustment by the computer of the size and opacity of the target image so that the target image remains detectable when the video camera is moved away from the screen.

16. The system of claim 1, further comprising a distance between the video camera and the screen that is estimated by the computer used in a computation of moving the target image, the distance used in a calculation to determine the movement of the target image.

17. The system of claim 1, further comprising a position of the video camera relative to the screen that is estimated by the computer based on one or more of yaw, pitch, and roll of the video camera, the position used in a calculation to determine the movement of the target image.

18. The system of claim 1, further comprising an initialization of the system that is manually performed by a user that picks the target image by pointing the video camera at the target image.

19. The system of claim 1, further comprising an initialization of the system that is automatically performed by a display of the target image onto different areas of the screen until the target image appears in a frame captured by the video camera.

20. The system of claim 1, further comprising an initialization of the system that is automatically performed by a prediction of where to move the target image by a recursive function that determines a first quadrant of the screen that appears in a frame captured by the video camera, then determines a second quadrant of the first quadrant that appears in the frame, and so on, until the quadrant size is equal to the target image size.

21. The system of claim 1, further comprising an initialization of the system that is automatically performed by a display of the target image to a predefined area on the screen such that the target image appears in the frame captured by the video camera pointed at the predefined area.

22. The system of claim 1, further comprising one or more additional screens such that the computer identifies where the screens are located in relation to each other.

23. The system of claim 22, wherein the identification of the screen locations in relation to each other is performed if the video camera that points to a first screen is moved to point to a second adjacent screen, and the target image follows from the first screen to the second screen.

24. The system of claim 1, further comprising one or more additional video cameras that control their respective cursors, each cursor associated with a target image, and each target image distinguished by one or more of color, shape, and texture.

13. A system for automatically and dynamically controlling a cursor on a screen when using a video camera as a pointing device, the system comprising:

a computer on which system software runs;

a screen connected to the computer such that the computer displays dynamic content to the screen;

a video camera connected to the computer such that the video camera points at the screen, and as the video camera films the screen, frames captured by the video camera are sent to the computer; and

a target image displayed by the computer onto the screen, such that frames captured by the video camera include the target image, and such that the computer dynamically moves the target image on the screen to ensure that the target image stays in the center of the view of the video camera, wherein the target image marks the position of the screen cursor of the video camera pointing device.

25. A program of instructions executable by a computer to perform a function for automatically and dynamically controlling a cursor on a screen when using a video camera as a pointing device, the function comprising the steps of:

displaying by the computer dynamic content to the screen connected to the computer;

filming of the screen by the video camera, the video camera pointing at the screen and connected to the computer;

displaying by the computer a target image to the screen such that frames captured by the video camera include the target image; and

dynamically moving the target image on the screen by the computer to ensure that the target image stays in the center of the view of the video camera, the target image marking the position of the screen cursor of the video camera pointing device.