Human-computer interface including haptically controlled interactions

• US 6,954,899 B1
• Filed: 02/16/2001
• Issued: 10/11/2005
• Est. Priority Date: 04/14/1997
• Status: Expired due to Term

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Novint Technologies, Inc.
• Inventors: Anderson, Thomas G.
• Primary Examiner(s): BAUTISTA, XIOMARA L
• Application Number:
  US09/785,696
• Time in Patent Office: 1,698 Days
• Field of Search: 345/702, 345/701, 345/700, 345/856, 345/862, 345/858, 345/861, 345/156, 345/163, 345/159, 345/157, 715700-702, 715/764, 715/765, 715/784, 715/786, 715/808, 715/828, 715/830, 715/856, 715/858, 715/861, 715/862, 715/973, 715/977
• US Class Current: 715/701
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed

Abstract

The present invention provides a method of human-computer interfacing that provides haptic feedback to control interface interactions such as scrolling or zooming within an application. Haptic feedback in the present method allows the user more intuitive control of the interface interactions, and allows the user'"'"'s visual focus to remain on the application. The method comprises providing a control domain within which the user can control interactions. For example, a haptic boundary can be provided corresponding to scrollable or scalable portions of the application domain. The user can position a cursor near such a boundary, feeling its presence haptically (reducing the requirement for visual attention for control of scrolling of the display). The user can then apply force relative to the boundary, causing the interface to scroll the domain. The rate of scrolling can be related to the magnitude of applied force, providing the user with additional intuitive, non-visual control of scrolling.

Claims

1. A method of providing user control of interactions in a computer display of an item, comprising:

a) Displaying a portion of the item;

b) Determining if a user-positioned cursor is within a threshold distance from an interaction boundary within the display, and if so, then;

i) applying a force to the cursor and communicating the force to the user;

ii) determining an input responsive force applied by the user to the input device;

iii) changing the portion of the item displayed, wherein the rate of change is determined from the input responsive force.

2. The method of claim 1 wherein changing the portion of the item displayed comprises changing the display to correspond to a portion adjacent the previous portion in the direction of the boundary.

3. The method of claim 1 wherein the item is a computer representation of a document.

4. The method of claim 3 wherein scrollable boundaries correspond to the top and bottom of the display, and wherein scrolling in the direction of the top boundary is disabled when the top of the document is displayed, and wherein scrolling in the direction of the bottom boundary is disabled when the bottom of the document is displayed.

5. The method of claim 1 wherein the rate of change increases with increasing input responsive force.

6. A computer-readable medium having stored thereon computer-executable instructions for performing the method of claim 1.

7. In a computer system providing display of a computer representation of a document, and user selection of the portion of the document to be displayed, a method of providing user control of scrolling among portions of the document comprising:

a) From the boundaries of the portion of the document displayed, determining which correspond to limits of the document;

b) For each boundary, establishing a haptic boundary in the range of motion of an input device approximately coincident with the visual boundary in the display;

c) Determining the position of a haptic cursor and, if the haptic cursor is near a haptic boundary that does not correspond to a limit of the document, then

i) Applying a force to the input device resisting motion of the input device toward said haptic boundary;

ii) Determining a user force applied by the user directed toward said haptic boundary;

iii) Scrolling the visual display of the document in the direction of said user force.

8. The method of claim 7, wherein the rate of scrolling is determined from the magnitude of the user force.

9. A computer-readable medium having stored thereon computer-executable instructions for performing the method of claim 7.

10. In a computer interface comprising a display and a haptic space, a method of providing user control of interactions, comprising:

a) Displaying a portion of an item;

b) Providing a control portion of the haptic space;

c) Determining if a user-positioned haptic cursor is within the control portion, and if so, then determining if the haptic cursor is within a threshold distance from a controllable boundary of the control portion, and if so, then;

i) applying a feedback force to a user input device affecting additional motion of the cursor relative to the boundary;

ii) determining an input force applied by the user to the input device;

iii) changing display of the item, wherein the rate of change is determined from the input force.

11. The method of claim 10, wherein the rate of change is determined from the magnitude of the input force.

12. The method of claim 10, wherein the control portion is activated responsive to direction of the user.

13. The method of claim 10, wherein determining if a user-positioned haptic cursor is within the control portion comprises moving the haptic cursor responsive to user control of an input device, and determining if such movement moves the haptic cursor within the control portion.

14. The method of claim 10, wherein determining if a user-positioned haptic cursor is within the control portion comprises detecting an indication from the user to move the cursor into the control portion.

15. The method of claim 10, wherein providing a control portion comprises providing haptic boundaries separating the control portion from the remainder of the haptic space.

16. The method of claim 15, wherein determining if a user-positioned haptic cursor is within the control portion comprises moving the haptic cursor responsive to user control of an input device, and determining if such movement moves the haptic cursor near a boundary, and if so, then applying a force to a user input device affecting further motion relative to said boundary.

17. A computer-readable medium having stored thereon computer-executable instructions for performing the method of claim 10.

18. In a computer interface comprising a display and a haptic space and adapted to display a document, a method of providing user control of scrolling the display of the document comprising:

a) Providing a scrolling zone portion of the haptic space, said portion disposed near an edge of the display of the document and extending from a first end to a second end oriented substantially parallel to the edge;

b) Determining the position of a user-controllable cursor in the haptic space, and, if the user cursor is within the scrolling zone portion and within a threshold distance of the first end or the second end, then applying a feedback force to a user input device resisting motion of the haptic cursor toward the nearest end, and determining the magnitude of a force applied by the user in opposition to said feedback force, and scrolling the display of the document in the direction according to the nearest edge at a rate determined from the magnitude of the user-applied force.

19. In a computer interface comprising a display and a haptic space and adapted to display a document, a method of providing user control of the display of the document comprising:

a) Detecting user activation of a control mode of interface;

b) When not in control mode, providing a computer interface suitable for interaction with the document;

c) When in control mode;

i) establishing haptic boundaries relative to the position of a user-controlled cursor when the user activated the control mode of the interface;

ii) determining the position of a user-controlled cursor in the haptic space, and, if the cursor is within a threshold distance of a haptic boundary, then applying a feedback force to a user input device affecting further motion of the cursor relative to said haptic boundary;

iii) determining the magnitude of a user-input force applied by the user to the user input device, and changing the display of the document in a direction according to the haptic boundary nearest the cursor at a rate determined from the magnitude of the user-input force.

20. A method of controlling interaction with a computer display of a document, comprising:

a) providing a three-dimensional control zone;

b) determining if the user indicates a transition into the control zone, and if so, then determining if the user positions a cursor near a boundary of the control zone, and if so, then determining an input force applied by the user to an input device and changing the display according to the direction and magnitude of the input force;

c) determining if the user indicates a transition out of the control zone, and if so, then providing interaction according to an application associated with the document.

21. A method as in claim 20, wherein:

a) the input device is moveable by the user in a three-dimensional space, characterized by x and y dimensions corresponding to a plane approximately parallel to the document displayed, and by a z dimension approximately orthogonal to said plane; and

b) the three-dimensional control zone comprises a portion of the three-dimensional space characterized by an entry region, defined by x, y, and z coordinates, and an active region, defined by x, y, and z coordinates, with the z coordinate not identical to the z coordinate of the entry region; and

c) determining if the user indicates a transition into the control zone comprises determining if the user has moved the input device from the entry region to the active region; and

d) determining if the user indicates a transition out of the control zone comprises determining if the user has moved the input device from the active region to the entry region.

22. A method as in claim 20, wherein:

a) the input device is moveable by the user in a three-dimensional space, characterized by x and y dimensions corresponding to a plane approximately parallel to the document displayed, and by a z dimension approximately orthogonal to said plane; and

b) a subset of the z dimension corresponds to a control portion; and

c) determining if the user indicates a transition into the control zone comprises determining if the user has moved the device such that the z coordinate of the device is within the control portion; and

d) determining if the user indicates a transition out of the control zone comprises determining if the user has moved the device such that the z coordinate of the device is not within the control portion.