Biomechanical user interface elements for pen-based computers

• US 7,663,605 B2
• Filed: 12/31/2003
• Issued: 02/16/2010
• Est. Priority Date: 01/08/2003
• Status: Active Grant

• Current Assignee: Alias Systems Corp. (Autodesk, Inc.), Autodesk, Inc.
• Original Assignee: Autodesk, Inc.
• Inventors: Fitzmaurice, George, Kurtenbach, Gordon
• Primary Examiner(s): Awad; Amr
• Assistant Examiner(s): Sherman; Stephen G
• Application Number:
  US10/748,684
• Publication Number:
  US 20040212605A1
• Time in Patent Office: 2,239 Days
• Field of Search: 345/184, 345/173, 345/179, 345/156, 715/700, 715/789, 715/811, 715/866
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04804 Transparency, e.g. transparent or translucent windows
• CPC Class Codes: G06F 2203/04807 Pen manipulated menu
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

The present invention is a system that uses natural user position and natural user motion to position and layout interface elements for a pen-based computer display. Graphical user interfaces, such as a slider or menu, are popped-up at a position convenient to the user, such as at the current position of the cursor. A rectilinear interface is oriented along a natural motion arc of the user, such as an elbow arc. An arc shaped interface can also be positioned along a natural motion arc, such as the elbow arc, and be shaped according the elbow are or be shaped by another natural motion arc such as a wrist arc of the user. The interface arc, whether shaping or orienting the interface, can be a single motion arc, such as an elbow arc, a composite arc of an elbow arc and a wrist arc, a sequence of an elbow arc and a wrist arc, a compound arc where an elbow arc blends into a wrist arc or an arc followed by a linear interface section.

Claims

1. A display located on a single side of a user, comprising:

an arc shaped control zone for a function of an interface located on the single side and at least a distance equal to a typical wrist away from a nearest edge of the display, the control zone forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the control zone having elements each having a weight and behaving like real-world materials;

an arc shaped interface element graphic located on the single side, aligned with the arc shaped control zone and indicating the function with the arc shaped interface graphic, and the arc shaped control zone aligned to a natural user motion produced by a compound motion of a rotation of the user elbow and rotation of a user wrist; and

an overflow control zone formed on a counter arc intersecting the arc shaped control zone at 90 degrees.

2. A display as recited in claim 1, wherein the alignment orients the graphic and zone with the motion.

3. A display as recited in claim 1, wherein the alignment follows the natural user motion.

4. A display as recited in claim 1, wherein the alignment positions the graphic and zone at a location accessible via the natural user motion.

5. A display as recited in claim 1, wherein the natural user motion comprises a curve determined by one or more strokes of the user on the display.

6. A display as recited in claim 5, wherein the curve includes natural motion variations.

7. A display as recited in claim 5, wherein the user natural motion stroke additionally comprises a finger motion curve, a shoulder motion curve and a combination of two or more of the curves.

8. A display as recited in claim 7, wherein the curve is a curve determined by a single user.

9. A display as recited in claim 1, further comprising an interface location at which the zone and graphic are positioned.

10. A display as recited in claim 9, wherein the interface location is specified by a cursor positioned by the user.

11. A graphical user interface, comprising:

a cursor positioned on a display by a user at a location;

a function control positioned on the display responsive to the location of the cursor, located on a single side of a user and at least a distance equal to a typical wrist away from a nearest edge of the display, having an arc shaped interface graphic indicating a function of the control and having an arc shape conforming to a motion arc of a hand caused by a compound motion of an arm about an elbow of the user and the hand about a wrist of the user, the function control forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the function control having elements each having a weight and behaving like real-world materials; and

an overflow control formed on a counter arc intersecting the function control at 90 degrees.

12. An interface as recited in claim 11, wherein the function control and overflow control comprise plural controls and the controls are aligned along each arc.

13. An interface as recited in claim 12, wherein a default control is positioned under the cursor at a particular instance.

14. An interface as recited in claim 12, wherein the controls can be one of re-oriented and moved.

15. An interface as recited in claim 12, wherein controls are oriented and shaped to conform to a wrist arc caused by a hand moving about a wrist of the user.

16. An interface as recited in claim 12, wherein the control is oriented to an extended arc.

17. An interface as recited in claim 11, wherein the control comprises plural controls and the shape of the sides of each of the controls is one of rectilinear, arc shaped, wedge shaped and triangular shaped.

18. An interface as recited in claim 11, further comprising an overflow interface positioned responsive to the motion arc.

19. An interface as recited in claim 11, wherein text of the control is rectilinear aligned with a display.

20. An interface as recited in claim 19, wherein the overflow interface is natural motion arc shaped.

21. An interface as recited in claim 11, wherein the cursor is positioned on a touch screen display and the compound motion is performed on the touch screen.

22. A graphical user interface, comprising:

a cursor positioned on a display by a user at a location; and

function controls positioned on the display responsive to the location of the cursor, at least a distance equal to a typical wrist away from a nearest edge of the display, having an interface graphic indicating a function of the controls and having an arc shape conforming to a motion arc of a hand caused by a compound motion of an arm about an elbow of the user and the hand about the wrist of the user, the function controls forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the controls having elements each having a weight and behaving like real-world materials, wherein a portion of the controls are aligned coincident to an arc intersecting the motion arc at 90 degrees.

23. A graphical user interface for a tablet personal computer having a stylus input system, comprising:

a cursor positioned on a display of the tablet personal computer by a user;

a function control positioned on the display at a location on the display designated by the stylus and at least a distance equal to a typical wrist away from an edge of the display responsive to the location of the cursor, having a interface graphic indicating a function of the control and having a graphic shape and position conforming to a natural motion arc of a hand caused by a compound motion of an arm about an elbow and of the hand moving about a wrist of the user, having plural controls with a default control positioned under the cursor, controls aligned coincident to the arc, the function control forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and function control having elements each having a weight and behaving like real-world materials, and controls aligned along a counter arc intersecting the motion arc at 90 degrees and where the controls are shaped responsive to the natural motion arc with natural variations; and

an overflow interface and shaped positioned responsive to the motion arc.

24. A method, comprising:

determining a position of a cursor as designated by a user;

positioning an arc shaped graphical user interface on a single side of a user and at least a distance equal to a typical wrist away from a nearest edge of the display, and responsive to the position where the arc of the shape is defined by a natural user motion caused by a compound motion of an arm about an elbow of the user and a hand about a wrist of the user, the arc shaped graphic interface having function controls forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the function controls having elements each having a weight and behaving like real-world materials; and

an overflow control zone formed on a counter arc shaped graphical interface intersecting the arc shaped graphical interface at 90 degrees.

25. A method as recited in claim 24, further comprising determining whether the user has specified a custom arc and positioning one of a custom and standard arc shaped interface responsive to the determination.

26. A computer readable storage for controlling a computer by determining a position of a cursor as designated by the user, positioning an arc shaped graphical user interface on a single side of a user and at least a distance equal to a typical wrist away from a nearest edge of the display, and responsive to the position where the arc of the shape is defined by a natural user motion caused by a compound motion of an arm about an elbow of the user and a hand about a wrist of the user, the arc shaped graphical interface having function controls forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the function controls having elements each having a weight and behaving like real-world materials, and positioning an overflow control zone formed on a counter arc shaped graphical interface intersecting the arc shaped graphical interface at 90 degrees.

27. A method, comprising:

allowing a user to make strokes with an input device caused by a compound motion of an arm about an elbow of the user and a hand about a wrist of the user with the input device located on a single side of a user;

determining an arc from the strokes;

laying out a first graphical user interface at least a distance equal to a typical wrist away from a nearest edge of a display, including controls, to conform to the arc, the controls forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the controls having elements each having a weight and behaving like real-world materials; and

laying out a second graphical user interface, including controls, formed on a counter arc intersecting the arc at 90 degrees.

28. A method as recited in claim 27, further comprising:

determining a position of a cursor specified by the user; and

positioning the interface responsive to the position; and

allowing the user to activate a function of the interface.

29. A method, comprising:

allowing a user to make strokes with an input device caused by a compound motion of an arm about an elbow of the user and a hand about a wrist of the user with the input device located on a single side of a user;

determining an arc from the strokes; and

laying out a graphical user interface, including controls, to conform to the arc,

wherein plural users are allowed to make strokes individually at different times and the arc is determined from a combination of strokes of the plural users.

30. An apparatus, comprising:

a display;

a computer producing an arc shaped graphical user interface on the display on a single side of a user and at least a distance equal to a typical wrist away from a nearest edge of the display, where the arc of the shape is defined by a natural user motion caused by a compound motion of an arm about an elbow of the user and a hand about a wrist of the user, the arc shaped graphical interface having function controls forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the function controls having elements each having a weight and behaving like real-world materials; and

the computer producing a counter arc shaped graphical user interface on the display intersecting the arc shaped graphical interface at 90 degrees and having an overflow control zone.

31. A display, comprising:

a control zone for a function of an interface on a single side of a user and located at least a distance equal to a typical wrist away from a nearest edge of the display, the control zone having function controls forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the function controls having elements each having a weight and behaving like real-world materials;

an interface element graphic aligned with the control zone and indicating the function with the interface graphic, and the control zone aligned to a natural user compound motion of an elbow motion and a wrist motion; and

an overflow control zone formed on a counter arc intersecting the control zone at 90 degrees.

32. A display as recited in claim 31, wherein a user natural motion stroke comprises one of an a wrist motion curve, a finger motion curve and a shoulder motion in combination with the elbow motion.

33. A display, comprising:

a control zone for a function of an interface and located at least a distance equal to a typical wrist away from a nearest edge of the display, the control zone forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the control zone having elements each having a weight and behaving like real-world materials;

an interface element graphic aligned with the control zone and indicating the function with the interface graphic, and the control zone only aligned to a natural user motion of independent finger motion; and

an overflow control zone formed on a counter arc and intersecting the control zone at 90 degrees.

34. A display as recited in claim 33, wherein the control zone further comprises a zone access comprising one of an elbow motion curve, a wrist motion curve and a shoulder motion curve in combination with the finger motion.

35. A display on a single side of a user, comprising:

a control zone for a function of an interface on the single side of a user and located at least a distance equal to a typical wrist away from a nearest edge of the display, the control zone forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the control zone having elements each having a weight and behaving like real-world materials;

an interface element graphic aligned with the control zone and indicating the function with the interface graphic, and the control zone aligned to a natural user motion of a shoulder motion; and

an overflow control zone formed on a counter arc and intersecting the control zone at 90 degrees.

36. A display as recited in claim 35, wherein the user natural motion stroke comprises one of an elbow motion curve, a wrist motion curve and a finger motion curve in combination with the shoulder motion.

37. A method, comprising:

determining a position of a cursor;

positioning an arc shaped graphical user interface on a single side of a user and at least a distance equal to a typical wrist away from a nearest edge of the display and responsive to the position where the arc is defined by a compound natural user motion of a hand when an arm is moved about an elbow of a user and about a wrist of the user, the arc shaped graphical interface having a control zone forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the control zone having elements each having a weight and behaving like real-world materials; and

positioning a counter arc shaped graphical interface to intersect the arc shaped control zone at 90 degrees, the counter arc shaped graphical interface having an overflow control zone.

38. A display, comprising:

an arc shaped control zone for a function of an interface and located at least a distance equal to a typical wrist away from a nearest edge of the display, the arc shaped control zone forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the control zone having elements each having a weight and behaving like real-world materials;

an arc shaped interface element graphic aligned with the arc shaped control zone and indicating the function with the arc shaped interface graphic, and the arc shaped control zone aligned to a natural user motion produced by only rotation of a user elbow; and

an overflow control zone formed on a counter arc shaped interface element graphic intersecting the arc shaped control zone at 90 degrees.

39. A display, comprising:

an arc shaped control zone for a function of an interface and located at least a distance equal to a typical wrist away from a nearest edge of the display, the arc shaped control zone forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the control zone having elements each having a weight and behaving like real-world materials;

an arc shaped interface element graphic aligned with the arc shaped control zone and indicating the function with the arc shaped interface graphic, and the arc shaped control zone aligned to a natural user motion produced by only rotation of a user wrist; and

an overflow control zone formed on a counter arc intersecting the arc shaped control zone at 90 degrees.

40. A display, comprising:

a control zone for a function of an interface and located at least a distance equal to a typical wrist away from a nearest edge of the display, the control zone forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the control zone having elements each having a weight and behaving like real-world materials;

an interface element graphic aligned with the control zone and indicating the function with the interface graphic, and the control zone aligned to a natural user motion of a static wrist and independent finger motion; and

an overflow control zone intersecting the control zone at 90 degrees.

41. A display, comprising:

an arc shaped control zone for a function of an interface and located at least a distance equal to a typical wrist away from a nearest edge of the display, the arc shaped control zone forming a command strip anchored at a base and responsive to manipulation including twisting, bending, pulling, and shrinking and the control zone having elements each having a weight and behaving like real-world materials;

an arc shaped interface element graphic aligned with the arc shaped control zone and indicating the function with the arc shaped interface graphic, and the arc shaped control zone aligned to a natural user motion produced by a concurrent motion of a rotation of the user elbow and rotation of a user wrist; and

an overflow control zone formed on a counter arc intersecting the arc shaped control zone at 90 degrees.