Human-computer interface including efficient three-dimensional controls

• US 20040207664A1
• Filed: 04/22/2004
• Published: 10/21/2004
• Est. Priority Date: 10/17/2000
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Novint Technologies, Inc.
• Inventors: Anderson, Thomas G.
• Granted Patent: US 7,379,048 B2
• Time in Patent Office: Days
• US Class Current: 715/851
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

The present invention provides a method of human-computer interfacing that provides efficient implementation of intuitive controls in a three-dimensional space. The method comprises providing a three-dimensional space, characterized by x and y dimensions, together defining a plane approximately orthogonal to the user'"'"'s direction of view into the space. A z dimension, approximately parallel to the direction of view, further characterizes the three-dimensional space. Objects in the space can be defined by x, y, and z coordinate sets. A control is provided having x and y coordinate sets as any other object. The z coordinate set of the control, however, spans a range of values, giving the control an increased depth. The control'"'"'s range of z can be infinite, making the control accessible at all depths, or can be a finite range, making the control accessible only from that range of depths. Movement of a cursor into the region corresponding to the control initiates interaction with the user according to the control. The increased depth makes it easier for the user to access the control, since it reduces the precision of depth perception required to find the control. Once the user is in the region corresponding to the control, the effective depth for interaction can be relative to the depth when the user entered the control region, providing control interaction independent of entering depth.

Claims

1. A method of providing a user-computer interface, comprising:

a) providing a display, characterized by x-display and y-display coordinates defining a plane approximately orthogonal to the direction of the user'"'"'s view into the display;

b) providing an input device, moveable by the user in x-device, y-device, and z-device dimensions;

c) establishing a correspondence between motion of the input device and motion of a cursor representation in the display;

d) displaying to the user a three-dimensional space using a first portion of the display plane;

e) displaying to the user a control using a second portion of the display plane;

f) determining if user motion of the input device corresponds to a cursor representation location within the first portion of the display plane, and, if so, then allowing user motion of the input device to move the cursor representation in three dimensions within the three-dimensional space;

g) determining if user motion of the input device corresponds to a cursor representation location within the second portion of the display plane, and, if so, then providing user interaction according to the control.

2. The method of claim 1 wherein interaction according to the control comprises two-dimensional interaction.

3. The method of claim 1 wherein the correspondence between motion of the input device and motion of a cursor representation in the display is such that motion of the input device into a portion of its range of motion defined by a first range of x-device coordinates, a second range of y-device coordinates, and all possible z-device coordinates, corresponds to a cursor representation within the second portion of the display.

4. The method of claim 1 wherein providing for interaction according to the control comprises:

a) establishing an initial control z coordinate from the z display coordinate of the cursor when the cursor entered the x, y region;

b) determining a control depth from the z component of the cursor'"'"'s motion after entering the control relative to the initial control z coordinate, where the control depth is substantially independent of the z coordinate when the cursor entered the x, y region; and

c) providing for interaction with the control according to the control depth.

5. The method of claim 4, wherein providing for interaction according to the control further comprises issuing a command to the computer if the control depth exceeds a threshold value.

6. The method of claim 4, wherein providing for interaction according to the control further comprises providing haptic feedback, comprising interaction relative to a haptic depth, to the user with the haptic depth corresponding to the control depth.

7. The method of claim 1, wherein providing for interaction according to the control comprises determining a command from the x and y coordinates of the cursor representation at the time the user indicates to issue a command.

8. The method of claim 7, wherein providing for interaction according to the control comprises issuing the command if a user indicates a cursor motion having a change in z coordinate relative to the display that exceeds a threshold value.

9. The method of claim 1, further comprising:

a) displaying a second control characterized by a third portion of the display; and

b) determining if user motion of the input device corresponds to a cursor representation location within the third portion of the display plane, and, if so, then providing user interaction according to the second control.

10. The method of claim 1, wherein displaying a control comprises displaying a region comprising a plurality of symbols representative of computer interface functions, and wherein providing for interaction according to the control comprises determining which symbol corresponds to the location of the cursor representation when the user indicates to initiate an action and initiating the interface function corresponding to the symbol.

11. The method of claim 1, wherein displaying the control comprises displaying the control at a first size when the cursor representation is not within a threshold distance of the second portion of the display, and displaying the control at a second size, larger than the first size, when the cursor representation is within the threshold distance of the second portion of the display.

12. A method of providing a human-computer interface, comprising:

a) Providing display of a three-dimensional space, where objects in the space are characterized by x, y, and z coordinates within the three-dimensional space, and where the display is provided relative to display x and y coordinates in a plane approximately orthogonal to the user direction of view into the display, and a z coordinate, approximately parallel to the user direction of view into the display, where the relationship between the three-dimensional space and the display coordinates is variable;

b) providing a control portion of the display, characterized by an x and y coordinate sets relative to the x and y coordinates of the display and independent of the relationship between the three-dimensional space and the display, and by a range of z coordinates relative to the z coordinates of the display and independent of the relationship between the three-dimensional space and the display, and providing for display of the control portion in conjunction with the display of the three-dimensional space;

c) providing an input device, moveable by the user in x-device, y-device, and z-device dimensions;

d) establishing a correspondence between motion of the input device and motion of a cursor representation in the display;

e) determining when the x and y coordinates, relative to the display, of the cursor representations are within the x and y coordinate sets of the control, and when the z coordinate, relative to the display, of the cursor representation is within the range of z coordinates for the control, and if so, issuing a command to the computer corresponding to the control responsive to a signal from the user.

13. The method of claim 12, wherein the signal from the user comprises the user directing a change in the z coordinate of the cursor representation exceeding a threshold value relative to the z coordinate of the cursor representation when its x and y coordinates first came within the x and y coordinate sets of the control.

14. The method of claim 12, further comprising:

a) a second control portion of the display, characterized by second x and y coordinate sets, and by a second range of z coordinates, and providing for display of the second control portion in conjunction with the display of the three-dimensional space;

b) determining when the x and y coordinates of the cursor representation are within the second x and y coordinate sets of the second control, and when the z coordinate of the cursor representation is within the second range of z coordinates for the second control, and if so, issuing a command to the computer corresponding to the second control responsive to a signal from the user.

15. The method of claim 12, further comprising providing haptic feedback to the user when the cursor representation is within the control portion.

16. The method of claim 15, wherein providing haptic feedback comprises providing a force resisting motion of the input device that would correspond to a change in the z coordinate of the cursor representation until the z coordinate change exceeds a threshold value, then perceptibly reducing the force.

17. The method of claim 12, further comprising providing a force resisting motion of the input device that would correspond with motion of the cursor representation toward a boundary between the control portion and the remainder of the space whenever the cursor representation approaches said boundary.

18. The method of claim 17, further comprising perceptibly reducing the force when the cursor representation crosses such boundary.