Camera assisted pen tablet

• US 20070024590A1
• Filed: 05/19/2005
• Published: 02/01/2007
• Est. Priority Date: 02/18/2004
• Status: Active Grant

• Current Assignee: Intellectual Ventures Assets 21 LLC (Intellectual Ventures LLC)
• Original Assignee: Rafal Krepec
• Inventors: Krepec, Rafal
• Granted Patent: US 7,969,409 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 5/08 Cursor circuits

Abstract

An x-y user input device, such as a pen tablet or digitizing tablet, used to control a cursor on a display of a data processing system, comprises an imaging device or camera and an image processor for displaying a captured image of a user'"'"'s hand operating the input device within a window of the display. The input device is generally operated using a pointing device that controls the cursor position on the display while being synchronously imaged and displayed with the user'"'"'s hand on the display. The position of the captured image of the user'"'"'s hand and pointing device in the window relative to the cursor corresponds to a position of the hand relative to the pointing device, thereby providing a visualization effect that increases user comfort and proficiency with the input device.

Claims

1. A method for displaying a user'"'"'s hand operating a pointing device controlling a cursor within a window of a display, the method comprising the steps of:

providing a substantially flat region on a surface, said region comprising dimensions proportionally the same as the window;

capturing an image of the hand operating the pointing device within said region; and

displaying said captured image in the window;

wherein a position of said captured image in the window relative to the cursor corresponds to a position of the hand relative to the pointing device.

2. The method as in claim 1, wherein the window comprises dimensions substantially the same as the display.

3. The method as in claim 1, wherein the window comprises dimensions less than the display.

4. The method as in claim 1, wherein the window comprises at least one sub-window.

5. The method as in claim 4, wherein said captured image is displayed only when the cursor is within at least one of said at least one sub-window.

6. The method as in claim 1, wherein said step of displaying said captured image comprises a step of mixing said captured image with a second image in the window.

7. The method as in claim 6, wherein said second image comprises a graphical user interface.

8. The method as in claim 6, wherein said step of mixing said captured image with a second image comprises selectively adjusting a translucence of said captured image such that when said captured image overlays a fraction of said second image, said fraction can still be seen through said captured image.

9. The method as in claim 6, wherein said step of mixing said captured image with a second image comprises selectively adjusting said position of said captured image to selectively align a point of said captured image with the cursor.

10. A user input device for displaying a user'"'"'s hand operating a pointing device controlling a cursor within a window of a display of a data processing system, the device comprising:

a substantially flat region on a surface, said region comprising dimensions proportionally the same as the window;

an imaging device for capturing an image of the hand operating the pointing device within said region; and

an image processor for displaying said captured image in the window;

wherein a position of said captured image in the window relative to the cursor corresponds to a position of the hand relative to the pointing device.

11. The user input device as in claim 10, wherein the window comprises dimensions substantially the same as the display.

12. The user input device as in claim 10, wherein the window comprises dimensions less than the display.

13. The user input device as in claim 10, wherein the window comprises at least one sub-window.

14. The user input device as in claim 13, wherein said captured image is displayed only when the cursor is within at least one of said at least one sub-window.

15. The user input device as in claim 10, wherein said image processor comprises an image mixer for combining said captured image with a second image.

16. The user input device as in claim 15, wherein said second image comprises a graphical user interface.

17. The user input device as in claim 15, wherein said image mixer comprises mixing software.

18. The user input device as in claim 17, wherein said mixing software comprises at least one adjustable image mixing parameter.

19. The user input device as in claim 18, wherein said at least one adjustable image mixing parameter comprises a translucence parameter for selectively adjusting a translucence of said captured image.

20. The user input device as in claim 18, wherein said at least one adjustable image mixing parameter comprises an alignment parameter for selectively adjusting said position of said captured image on the display.

21. The user input device as in claim 17, wherein said software is implemented in the data processing system.

22. The user input device as in claim 15, wherein said image mixer comprises mixing hardware.

23. The user input device as in claim 22, wherein said mixing hardware comprises at least one adjustable image mixing switch.

24. The user input device as in claim 10, wherein said at least one adjustable image mixing switch comprises a translucence switch for selectively adjusting a translucence of said captured image.

25. The user input device as in claim 10, wherein said at least one adjustable image mixing switch comprises an alignment switch for selectively adjusting said position of said captured image on the display.

26. The user input device as in claim 10, wherein said captured image is translucent.

27. The user input device as in claim 10, wherein said captured image is opaque.

28. The user input device as in claim 10, wherein said imaging device is an optical device.

29. The user input device as in claim 28, wherein said optical device is a camera.

30. The user input device as in claim 29 wherein said camera is a CCD camera.

31. The user input device as in claim 29 wherein said camera is a CMOS camera.

32. The user input device as in claim 10, wherein said imaging device is a heat sensing device.

33. The user input device as in claim 10, wherein said imaging device is a sonar device.

34. The user input device as in claim 10, wherein said imaging device is a laser scanning device.

35. The user input device as in claim 10, wherein said imaging device is an IR imaging device.

36. The user input device as in claim 10, wherein said imaging device is positioned above said region.

37. The user input device as in claim 36, wherein a position of said imaging device is substantially centered above said region.

38. The user input device as in claim 36, wherein an optical axis of said imaging device is substantially normal to said surface.

39. The user input device as in claim 10, wherein said pointing device is a pen-like object.

40. The user input device as in claim 10, wherein said pointing device is a stylus.

41. The user input device as in claim 10, wherein said pointing device is a mouse.

42. The user input device as in claim 10, wherein said pointing device is a user finger.