Human-computer interface including efficient three-dimensional controls

• US 6,727,924 B1
• Filed: 10/17/2000
• Issued: 04/27/2004
• Est. Priority Date: 10/17/2000
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Novint Technologies, Inc.
• Inventors: Anderson, Thomas G.
• Primary Examiner(s): Cabeca, John W.
• Assistant Examiner(s): DETWILER, BRIAN J
• Application Number:
  US09/690,343
• Time in Patent Office: 1,288 Days
• Field of Search: 345/782, 345/836, 345/848, 345/849, 345/850, 345/851, 345/852
• US Class Current: 715/851
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

A method of human-computer interfacing provides efficient intuitive controls in a three-dimensional space. The method provides a three-dimensional space, characterized by a z dimension approximately parallel to the user'"'"'s direction of view, and x and y dimensions approximately orthogonal thereto. A control has x and y coordinate sets, and a z coordinate set that spans a range of values. The range can be infinite, making the control accessible at all depths, or can be a finite range, making the control accessible from that range of depths. Movement of a cursor into the control'"'"'s region initiates user interaction according to the control. The control depth reduces the precision of depth perception required to find the control. Once the user is in the control region, the effective depth for interaction can be relative to the depth when the user entered, providing control interaction independent of entering depth.

Claims

1. A method of providing a user-computer interface a three-dimensional space characterized by x, y, and z space coordinates, using a display characterized by x and y display coordinates in a plane approximately orthogonal to the direction of the user'"'"'s view into the display, and by z display coordinate approximately parallel to the direction of the users view into the display, wherein the three-dimensional space is moveable, translatable, or both relative to the x, y, and z display coordinates, comprising:

a) displaying the space;

b) displaying a control characterized by an x, y region of the display coordinates, and by a z display coordinate region that defines where the control can be accessed, wherein the x, y, and z regions are independent of motion and translation of the three-dimensional space relative to the display coordinates;

c) determining if the user indicates a cursor positioned within the x, y region, and at a z coordinate within the z region, and if so then providing for interaction with the user according to the control and according to changes in the z coordinate of the cursor position, and if not then providing for three-dimensional interaction with the user according to the three-dimensional space.

2. The method of claim 1 wherein interaction according to the control comprises two-dimensional interaction.

3. The method of claim 1 wherein the range of z dimensions for the control corresponds to the entire range of z coordinates of the display.

4. The method of claim 1 wherein providing for interaction according to the control comprises:

a) establishing an initial control z coordinate from the z display coordinate of the cursor when the cursor entered the x, y region;

b) determining a control depth from the z component of the cursor'"'"'s motion after entering the control relative to the initial control z coordinate, where the control depth is substantially independent of the z coordinate when the cursor entered the x, y region; and

c) providing for interaction with the control according to the control depth.

5. The method of claim 4, wherein providing for interaction according to the control further comprises issuing a command to the computer if the control depth exceeds a threshold value.

6. The method of claim 4, wherein providing for interaction according to the control further comprises providing haptic feedback, comprising interaction relative to a haptic depth, to the user with the haptic depth corresponding to the control depth.

7. The method of claim 1, wherein providing for interaction according to the control comprises determining a command from the x and y coordinates of the cursor at the time the user indicates to issue a command.

8. The method of claim 7, wherein providing for interaction according to the control comprises issuing the command if a user indicates a cursor motion having a change in z coordinate relative to the display that exceeds a threshold value.

9. The method of claim 1, further comprising:

a) displaying a second control characterized by a second x, y region of the display, and by a second z display region; and

b) determining if the user indicates a cursor positioned within the second x, y region, and at a z coordinate within the second z region, and if so then providing for interaction with the user according to the second control.

10. The method of claim 9, wherein the x, y ranges of the two controls partially overlap.

11. The method of claim 1, wherein displaying a control comprises displaying a region comprising a plurality of symbols representative of computer interface functions, and wherein providing for interaction according to the control comprises determining which symbol corresponds to the location of the cursor when the user indicates to initiate an action and initiating the interface function corresponding to the symbol.

12. The method of claim 1, wherein displaying the control comprises displaying the control at a first size when the cursor is not within the x, y region and the z region, and displaying the control at a second size, larger than the first size, when the cursor is within a threshold distance of the x, y region and the z region.

13. A method of providing a user-computer interface to a three-dimensional space characterized by x and y dimensions in a plane approximately orthogonal to the direction of the user'"'"'s view into the space, and by z dimension approximately parallel to the direction of the user'"'"'s view into the space, comprising:

a) displaying the space;

b) displaying a control characterized by an x, y region of the display, and by a z coordinate set that extends over a range;

c) determining if the user indicates a cursor positioned within the x, y region, and at a z coordinate within the z coordinate set range, and if so then providing for interaction with the user according to the control, and if not then providing for three-dimensional interaction with the user according to the three-dimensional space;

wherein providing for interaction according to the control comprises;

d) providing a haptic component of interaction with a reference-haptic depth corresponding to the z coordinate of the cursor when the user entered the x, y region;

e) providing a visual component of interaction with the visual z coordinate having an initial value independent of the z coordinate of the cursor when the user entered the x, y region.

14. The method of claim 13 wherein the haptic component of interaction has a lesser range of depth than the depth of the three-dimensional space.

15. The method of claim 13 wherein providing a haptic component of interaction comprises changing the reference haptic depth responsive to user change of the z coordinate of the cursor in a direction toward a desirable portion of the range of motion of the cursor control device.

16. A method of providing a human-computer interface, comprising:

a) providing display of a three-dimensional space, where objects in the space are characterized by x, y, and z coordinates within the three-dimensional space, and where the display is provided relative to display x and y coordinates in a plane approximately orthogonal to the user direction of view into the display, and a z coordinate, approximately parallel to the user direction of view into the display, where the relationship between the three-dimensional space and the display coordinates is variable;

b) providing a control portion of the display, characterized by an x and y coordinate sets relative to the x and y coordinates of the display and independent of the relationship between the three-dimensional space and the display, and by a range of z coordinates relative to the z coordinates of the display and independent of the relationship between the three-dimensional space and the display, and providing for display of the control portion in conjunction with the display of the three-dimensional space;

c) determining when the x and y coordinates, relative to the display, of a cursor are within the x and y coordinate sets of the control, and when the z coordinate, relative to the display, of the cursor is within the range of z coordinates for the control, and if so, issuing a command to the computer corresponding to the control responsive to a signal from the user, which signal includes information derived from change in the z coordinate of the cursor.

17. The method of claim 16, wherein the signal from the user comprises the user directing a change in the z coordinate of the cursor exceeding a threshold value relative to the z coordinate of the cursor when its x and y coordinates first came within the x and y coordinate sets of the control.

18. The method of claim 16, further comprising:

a) a second control portion of the display, characterized by second x and y coordinate sets, and by a second range of z coordinates, and providing for display of the second control portion in conjunction with the display of the three-dimensional space;

b) determining when the x and y coordinates of a cursor are within the second x and y coordinate sets of the second control, and when the z coordinate of the cursor is within the second range of z coordinates for the second control, and if so, issuing a command to the computer corresponding to the second control responsive to a signal from the user.

19. The method of claim 16, further comprising providing haptic feedback to the user when the cursor is within the control portion.

20. The method of claim 19, wherein providing haptic feedback comprises providing a force resisting a change in the z coordinate of the cursor until the z coordinate change exceeds a threshold value, then perceptibly reducing the force.

21. The method of claim 16, further comprising providing a force directed substantially away from a boundary between the control portion and the remainder of the space whenever the cursor approaches said boundary.

22. The method of claim 21, further comprising perceptibly reducing the force when the cursor crosses such boundary.