BIMANUAL INTERACTIONS ON DIGITAL PAPER USING A PEN AND A SPATIALLY-AWARE MOBILE PROJECTOR

• US 20110216091A1
• Filed: 03/03/2011
• Published: 09/08/2011
• Est. Priority Date: 03/04/2010
• Status: Active Grant

• Current Assignee: Autodesk, Inc.
• Original Assignee: Autodesk, Inc.
• Inventors: GROSSMAN, Tovi, FITZMAURICE, George, GUIMBRETIERE, Francois V., SONG, Hyunyoung
• Granted Patent: US 9,128,537 B2
• Time in Patent Office: Days
• US Class Current: 345/634
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/03542 Light pens for emitting or receiving light
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

One embodiment of the present invention sets forth a technique for providing an end user with a spatially-aware projector combined with a spatially-aware digital pen for use in a design environment. Paper documents are augmented to allow a user to access additional information and computational tools through projected interfaces. Virtual ink may be managed in single and multi-user environments to enhance collaboration and data management. The spatially-aware projector pen provides end-users with dynamic visual feedback and improved interaction capabilities.

Claims

1. A computer-implemented method for configuring a spatially-aware projector to output a projected image, the method comprising:

receiving a first position of the spatially-aware projector in a display surface from a first position tracking mechanism included within the spatially-aware projector;

retrieving design data from a memory based on the first position; and

generating a projected image based on the design data for display on the display surface.

2. The method of claim 1, further comprising receiving input from a scroll-wheel included in the spatially-aware projector that, when rotated, causes the size of at least one element in the projected image to change.

3. The method of claim 1, further comprising receiving a second position of a spatially-aware digital pen in a two-dimensional space from a second position tracking mechanism included within the spatially-aware digital pen.

4. The method of claim 3, further comprising retrieving either additional design data or image data from the memory based on the second position.

5. The method of claim 4, wherein the spatially-aware digital pen includes a force sensor configured to detect when a tip of the spatially-aware digital pen contacts the display surface.

6. The method of claim 5, further comprising receiving data from the spatially-aware pen in the form of virtual ink when the digital pen contacts and moves across the display surface.

7. The method of claim 6, further comprising storing the virtual ink as image data in the memory.

8. The method of claim 5, further comprising determining that the spatially-aware digital pen is contact with and held in place against the display surface for a threshold amount of time, and updating the projected image to include a radial user interface menu that includes a plurality of regions separated by boundaries, wherein each of the regions represents a different menu item included in the radial menu.

9. The method of claim 8, further comprising determining that a first region in the plurality of regions has been selected, and executing an operation associated with the first region.

10. The method of claim 9, wherein the first region is selected when the spatially-aware projector is relocated and the spatially-aware digital pen remains fixed and in contact with the display surface causing a boundary delineating the first region to be crossed by a virtual cursor that is controlled by movements of the spatially-aware digital pen.

11. The method of claim 5, further comprising:

receiving, from the spatially-aware digital pen, a selection of an element that is outside of the boundaries of the projected image to produce a copy of the element;

generating a paste position within the display surface based on the first position of the spatially-aware projector; and

updating the projected image to display the copy of the element at the paste position within the projected image.

12. A computer-readable storage medium storing instructions that, when executed by a processor, cause a spatially-aware projector to output a projected image, the method comprising:

receiving a first position of the spatially-aware projector in a display surface from a first position tracking mechanism included within the spatially-aware projector;

retrieving design data from a memory based on the first position; and

generating a projected image based on the design data for display on the display surface.

13. The computer-readable storage medium of claim 12, further comprising receiving input from a scroll-wheel included in the spatially-aware projector that, when rotated, causes the size of at least one element in the projected image to change.

14. The computer-readable storage medium of claim 12, further comprising receiving a second position of a spatially-aware digital pen in a two-dimensional space from a second position tracking mechanism included within the spatially-aware digital pen.

15. The computer-readable storage medium of claim 14, further comprising retrieving either additional design data or image data from the memory based on the second position.

16. The computer-readable storage medium of claim 15, wherein the spatially-aware digital pen includes a force sensor configured to detect when a tip of the spatially-aware digital pen contacts the display surface.

17. The computer-readable storage medium of claim 16, further comprising receiving data from the spatially-aware pen in the form of virtual ink when the digital pen contacts and moves across the display surface.

18. The computer-readable storage medium of claim 17, further comprising storing the virtual ink as image data in the memory.

19. The computer-readable storage medium of claim 16, further comprising determining that the spatially-aware digital pen is contact with and held in place against the display surface for a threshold amount of time, and updating the projected image to include a radial user interface menu that includes a plurality of regions separated by boundaries, wherein each of the regions represents a different menu item included in the radial menu.

20. The computer-readable storage medium of claim 16, further comprising:

receiving, from the spatially-aware digital pen, a selection of an element that is outside of the boundaries of the projected image to produce a copy of the element;

generating a paste position within the display surface based on the first position of the spatially-aware projector; and

updating the projected image to display the copy of the element at the paste position within the projected image.